[ 461.910446] nova-conductor[52380]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. [ 463.115578] nova-conductor[52380]: DEBUG oslo_db.sqlalchemy.engines [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52380) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 463.143652] nova-conductor[52380]: DEBUG nova.context [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),b5105805-148c-4545-8985-397f7b32e247(cell1) {{(pid=52380) load_cells /opt/stack/nova/nova/context.py:464}} [ 463.145504] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52380) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 463.145708] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52380) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 463.146166] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52380) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 463.146499] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52380) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 463.146692] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52380) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 463.147606] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52380) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 463.152814] nova-conductor[52380]: DEBUG oslo_db.sqlalchemy.engines [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52380) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 463.153187] nova-conductor[52380]: DEBUG oslo_db.sqlalchemy.engines [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52380) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 463.212189] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Acquiring lock "singleton_lock" {{(pid=52380) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 463.212365] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Acquired lock "singleton_lock" {{(pid=52380) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 463.212586] nova-conductor[52380]: DEBUG oslo_concurrency.lockutils [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Releasing lock "singleton_lock" {{(pid=52380) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 463.213064] nova-conductor[52380]: INFO oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Starting 2 workers [ 463.217340] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Started child 52799 {{(pid=52380) _start_child /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:575}} [ 463.222365] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Started child 52800 {{(pid=52380) _start_child /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:575}} [ 463.222838] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Full set of CONF: {{(pid=52380) wait /usr/local/lib/python3.10/dist-packages/oslo_service/service.py:649}} [ 463.223033] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ******************************************************************************** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}} [ 463.223185] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] Configuration options gathered from: {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}} [ 463.223362] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] command line args: ['--config-file', '/etc/nova/nova.conf'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}} [ 463.223626] nova-conductor[52799]: INFO nova.service [-] Starting conductor node (version 0.1.0) [ 463.223880] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] config files: ['/etc/nova/nova.conf'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}} [ 463.223880] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ================================================================================ {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}} [ 463.224209] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] allow_resize_to_same_host = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.224447] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] arq_binding_timeout = 300 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.224508] nova-conductor[52800]: INFO nova.service [-] Starting conductor node (version 0.1.0) [ 463.224685] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] block_device_allocate_retries = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.224867] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] block_device_allocate_retries_interval = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.225095] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cert = self.pem {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.225284] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute_driver = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.225521] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute_monitors = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.225766] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] config_dir = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.225970] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] config_drive_format = iso9660 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.226127] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] config_file = ['/etc/nova/nova.conf'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.226307] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] config_source = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.226501] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] console_host = devstack {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.226724] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] control_exchange = nova {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.226929] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cpu_allocation_ratio = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.227124] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] daemon = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.227317] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] debug = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.227496] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] default_access_ip_network_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.227681] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] default_availability_zone = nova {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.227871] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] default_ephemeral_format = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.228169] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.228356] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] default_schedule_zone = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.228513] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] disk_allocation_ratio = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.228688] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] enable_new_services = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.228926] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] enabled_apis = ['osapi_compute'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.229133] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] enabled_ssl_apis = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.229323] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] flat_injected = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.229477] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] force_config_drive = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.229646] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] force_raw_images = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.229840] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] graceful_shutdown_timeout = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.230038] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] heal_instance_info_cache_interval = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.230480] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] host = devstack {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.230691] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] initial_cpu_allocation_ratio = 4.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.230862] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] initial_disk_allocation_ratio = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.231047] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] initial_ram_allocation_ratio = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.231301] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.231458] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instance_build_timeout = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.231612] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instance_delete_interval = 300 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.231772] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instance_format = [instance: %(uuid)s] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.231952] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instance_name_template = instance-%08x {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.232122] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instance_usage_audit = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.232307] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instance_usage_audit_period = month {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.232484] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.232668] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] instances_path = /opt/stack/data/nova/instances {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.232830] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] internal_service_availability_zone = internal {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.232998] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] key = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.233169] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] live_migration_retry_count = 30 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.233344] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_config_append = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.233519] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.233675] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_dir = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.233834] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.233990] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_options = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.234195] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_rotate_interval = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.234390] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_rotate_interval_type = days {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.234571] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] log_rotation_type = none {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.234717] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.234846] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.235018] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.235202] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.235327] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.235530] nova-conductor[52799]: DEBUG oslo_db.sqlalchemy.engines [None req-710c2686-b3d7-4de5-9cbb-0621dc6eac7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52799) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 463.235584] nova-conductor[52800]: DEBUG oslo_db.sqlalchemy.engines [None req-76b4a051-480a-43ec-bdcd-da762d1dff24 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52800) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 463.235624] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] long_rpc_timeout = 1800 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.235688] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] max_concurrent_builds = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.235834] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] max_concurrent_live_migrations = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.235987] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] max_concurrent_snapshots = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.236154] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] max_local_block_devices = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.236304] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] max_logfile_count = 30 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.236483] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] max_logfile_size_mb = 200 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.236678] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] maximum_instance_delete_attempts = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.236881] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] metadata_listen = 0.0.0.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.237105] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] metadata_listen_port = 8775 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.237279] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] metadata_workers = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.237440] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] migrate_max_retries = -1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.237607] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] mkisofs_cmd = genisoimage {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.237850] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] my_block_storage_ip = 10.180.1.21 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.237985] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] my_ip = 10.180.1.21 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.238160] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] network_allocate_retries = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.238355] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.238537] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] osapi_compute_listen = 0.0.0.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.238698] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] osapi_compute_listen_port = 8774 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.238880] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] osapi_compute_unique_server_name_scope = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.239057] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] osapi_compute_workers = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.239218] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] password_length = 12 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.239373] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] periodic_enable = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.239525] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] periodic_fuzzy_delay = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.239684] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] pointer_model = usbtablet {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.239866] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] preallocate_images = none {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.240031] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] publish_errors = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.240176] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] pybasedir = /opt/stack/nova {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.240326] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ram_allocation_ratio = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.240478] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rate_limit_burst = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.240641] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rate_limit_except_level = CRITICAL {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.240797] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rate_limit_interval = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.240946] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] reboot_timeout = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.241107] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] reclaim_instance_interval = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.241254] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] record = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.241402] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] reimage_timeout_per_gb = 20 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.241551] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] report_interval = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.241703] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rescue_timeout = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.241862] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] reserved_host_cpus = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.242034] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] reserved_host_disk_mb = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.242191] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] reserved_host_memory_mb = 512 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.242355] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] reserved_huge_pages = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.242512] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] resize_confirm_window = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.242666] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] resize_fs_using_block_device = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.242825] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] resume_guests_state_on_host_boot = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.243013] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.243180] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rpc_response_timeout = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.243355] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] run_external_periodic_tasks = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.243532] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] running_deleted_instance_action = reap {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.243691] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] running_deleted_instance_poll_interval = 1800 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.243848] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] running_deleted_instance_timeout = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.243998] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler_instance_sync_interval = 120 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.244170] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_down_time = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.244350] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] servicegroup_driver = db {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.244503] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] shelved_offload_time = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.244683] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] shelved_poll_interval = 3600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.244878] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] shutdown_timeout = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.245047] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] source_is_ipv6 = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.245208] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ssl_only = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.245384] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] state_path = /opt/stack/data/nova {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.245539] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] sync_power_state_interval = 600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.245695] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] sync_power_state_pool_size = 1000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.245852] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] syslog_log_facility = LOG_USER {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.246021] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] tempdir = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.246189] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] timeout_nbd = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.246366] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] transport_url = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.246522] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] update_resources_interval = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.246698] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] use_cow_images = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.246884] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] use_eventlog = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.247052] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] use_journal = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.247213] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] use_json = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.247366] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] use_rootwrap_daemon = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.247546] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] use_stderr = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.247761] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] use_syslog = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.247951] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vcpu_pin_set = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.248141] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vif_plugging_is_fatal = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.248328] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vif_plugging_timeout = 300 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.248529] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] virt_mkfs = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.248721] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] volume_usage_poll_interval = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.248893] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] watch_log_file = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.249092] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] web = /usr/share/spice-html5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} [ 463.249383] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_concurrency.disable_process_locking = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.249573] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.249766] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.249931] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.250111] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.250396] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.250469] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.250678] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.auth_strategy = keystone {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.250868] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.compute_link_prefix = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.251062] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.251236] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.dhcp_domain = novalocal {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.251404] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.enable_instance_password = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.251562] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.glance_link_prefix = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.251726] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.251910] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.instance_list_cells_batch_strategy = distributed {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.252100] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.instance_list_per_project_cells = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.252263] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.list_records_by_skipping_down_cells = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.252417] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.local_metadata_per_cell = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.252583] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.max_limit = 1000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.252763] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.metadata_cache_expiration = 15 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.252944] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.neutron_default_tenant_id = default {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.253108] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.use_forwarded_for = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.253299] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.use_neutron_default_nets = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.253465] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.253620] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.vendordata_dynamic_failure_fatal = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.253799] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.253967] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.vendordata_dynamic_ssl_certfile = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.254148] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.vendordata_dynamic_targets = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.254324] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.vendordata_jsonfile_path = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.254500] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api.vendordata_providers = ['StaticJSON'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.254778] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.backend = dogpile.cache.memcached {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.254954] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.backend_argument = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.255158] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.config_prefix = cache.oslo {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.255343] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.dead_timeout = 60.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.255507] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.debug_cache_backend = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.255679] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.enable_retry_client = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.255837] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.enable_socket_keepalive = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.256014] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.enabled = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.256244] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.expiration_time = 600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.256522] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.hashclient_retry_attempts = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.256727] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.hashclient_retry_delay = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.256897] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_dead_retry = 300 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.257075] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_password = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.257235] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.257700] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.257700] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_pool_maxsize = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.257700] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.257818] nova-conductor[52799]: DEBUG nova.service [None req-710c2686-b3d7-4de5-9cbb-0621dc6eac7e None None] Creating RPC server for service conductor {{(pid=52799) start /opt/stack/nova/nova/service.py:182}} [ 463.257854] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_sasl_enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.258025] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.258215] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_socket_timeout = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.258395] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.memcache_username = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.258561] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.proxies = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.258722] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.retry_attempts = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.258882] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.retry_delay = 0.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.259050] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.socket_keepalive_count = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.259210] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.socket_keepalive_idle = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.259367] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.socket_keepalive_interval = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.259518] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.tls_allowed_ciphers = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.259629] nova-conductor[52800]: DEBUG nova.service [None req-76b4a051-480a-43ec-bdcd-da762d1dff24 None None] Creating RPC server for service conductor {{(pid=52800) start /opt/stack/nova/nova/service.py:182}} [ 463.259671] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.tls_cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.259812] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.tls_certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.259970] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.tls_enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.260132] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cache.tls_keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.260337] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.260521] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.auth_type = password {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.260697] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.260920] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.catalog_info = volumev3::publicURL {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.261092] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.261257] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.261432] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.cross_az_attach = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.261594] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.debug = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.261747] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.endpoint_template = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.261930] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.http_retries = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.262109] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.262268] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.262449] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.os_region_name = RegionOne {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.262608] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.262765] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cinder.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.262931] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.263097] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.cpu_dedicated_set = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.263250] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.cpu_shared_set = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.263409] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.image_type_exclude_list = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.263572] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.live_migration_wait_for_vif_plug = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.263731] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.max_concurrent_disk_ops = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.263892] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.max_disk_devices_to_attach = -1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.264062] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.264226] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.264383] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.resource_provider_association_refresh = 300 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.264548] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.shutdown_retry_interval = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.264743] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.264919] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] conductor.workers = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.265107] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] console.allowed_origins = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.265263] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] console.ssl_ciphers = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.265431] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] console.ssl_minimum_version = default {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.265596] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] consoleauth.token_ttl = 600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.265785] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.265941] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.266128] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.266284] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.connect_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.266455] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.connect_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.266609] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.endpoint_override = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.266790] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.266949] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.267121] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.max_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.267278] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.min_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.267428] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.region_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.267580] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.service_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.267772] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.service_type = accelerator {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.267942] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.268109] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.status_code_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.268262] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.status_code_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.268415] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.268588] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.268745] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] cyborg.version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.268936] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.backend = sqlalchemy {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.269179] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.connection = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.269354] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.connection_debug = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.269526] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.connection_parameters = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.269688] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.connection_recycle_time = 3600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.269873] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.connection_trace = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.270046] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.db_inc_retry_interval = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.270208] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.db_max_retries = 20 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.270367] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.db_max_retry_interval = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.270521] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.db_retry_interval = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.270687] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.max_overflow = 50 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.270848] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.max_pool_size = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.271045] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.max_retries = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.271225] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.mysql_enable_ndb = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.271396] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.271572] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.mysql_wsrep_sync_wait = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.271731] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.pool_timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.271899] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.retry_interval = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.272064] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.slave_connection = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.272234] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.sqlite_synchronous = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.272394] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] database.use_db_reconnect = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.272572] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.backend = sqlalchemy {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.272763] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.connection = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.272955] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.connection_debug = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.273139] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.connection_parameters = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.273298] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.connection_recycle_time = 3600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.273460] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.connection_trace = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.273613] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.db_inc_retry_interval = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.273769] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.db_max_retries = 20 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.273929] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.db_max_retry_interval = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.274093] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.db_retry_interval = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.274258] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.max_overflow = 50 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.274412] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.max_pool_size = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.274572] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.max_retries = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.274727] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.mysql_enable_ndb = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.274887] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.mysql_sql_mode = TRADITIONAL {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.275049] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.mysql_wsrep_sync_wait = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.275208] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.pool_timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.275367] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.retry_interval = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.275518] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.slave_connection = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.275679] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] api_database.sqlite_synchronous = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.275869] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] devices.enabled_mdev_types = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.276053] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.276212] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ephemeral_storage_encryption.enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.276373] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ephemeral_storage_encryption.key_size = 512 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.276562] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.api_servers = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.276750] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.276915] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.277087] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.277243] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.connect_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.277394] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.connect_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.277567] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.debug = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.277801] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.default_trusted_certificate_ids = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.277969] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.enable_certificate_validation = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.278141] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.enable_rbd_download = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.278294] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.endpoint_override = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.278454] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.278685] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.278981] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.max_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.279215] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.min_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.279487] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.num_retries = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.279777] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.rbd_ceph_conf = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.280080] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.rbd_connect_timeout = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.280370] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.rbd_pool = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.280661] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.rbd_user = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.280943] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.region_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.281234] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.service_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.281530] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.service_type = image {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.281810] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.282104] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.status_code_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.282382] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.status_code_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.282651] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.282963] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.valid_interfaces = ['internal', 'public'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.283275] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.verify_glance_signatures = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.283552] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] glance.version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.283829] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] guestfs.debug = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.284155] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.config_drive_cdrom = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.284398] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.config_drive_inject_password = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.284636] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.dynamic_memory_ratio = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.284898] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.enable_instance_metrics_collection = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.285106] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.enable_remotefx = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.285286] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.instances_path_share = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.285454] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.iscsi_initiator_list = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.285493] nova-conductor[52799]: DEBUG nova.service [None req-710c2686-b3d7-4de5-9cbb-0621dc6eac7e None None] Join ServiceGroup membership for this service conductor {{(pid=52799) start /opt/stack/nova/nova/service.py:199}} [ 463.285616] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.limit_cpu_features = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.285760] nova-conductor[52799]: DEBUG nova.servicegroup.drivers.db [None req-710c2686-b3d7-4de5-9cbb-0621dc6eac7e None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=52799) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 463.285815] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.285929] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.286101] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.power_state_check_timeframe = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.286260] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.power_state_event_polling_interval = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.286443] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.286603] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.use_multipath_io = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.286783] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.volume_attach_retry_count = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.286946] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.volume_attach_retry_interval = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.287113] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.vswitch_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.287277] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.287443] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] mks.enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.288041] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.288251] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] image_cache.manager_interval = 2400 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.288433] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] image_cache.precache_concurrency = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.288597] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] image_cache.remove_unused_base_images = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.288759] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.288924] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.289132] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] image_cache.subdirectory_name = _base {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.289332] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.api_max_retries = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.289492] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.api_retry_interval = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.289646] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.289808] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.auth_type = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.289961] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.290130] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.290286] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.290461] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.connect_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.290613] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.connect_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.290779] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.endpoint_override = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.290939] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.291104] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.291256] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.max_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.291403] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.min_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.291551] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.partition_key = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.291709] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.peer_list = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.291747] nova-conductor[52800]: DEBUG nova.service [None req-76b4a051-480a-43ec-bdcd-da762d1dff24 None None] Join ServiceGroup membership for this service conductor {{(pid=52800) start /opt/stack/nova/nova/service.py:199}} [ 463.291858] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.region_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.291969] nova-conductor[52800]: DEBUG nova.servicegroup.drivers.db [None req-76b4a051-480a-43ec-bdcd-da762d1dff24 None None] DB_Driver: join new ServiceGroup member devstack to the conductor group, service = {{(pid=52800) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}} [ 463.292027] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.serial_console_state_timeout = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.292169] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.service_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.292348] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.service_type = baremetal {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.292503] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.292652] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.status_code_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.292801] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.status_code_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.292951] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.293138] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.valid_interfaces = ['internal', 'public'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.293295] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ironic.version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.293498] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.293687] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] key_manager.fixed_key = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.293897] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.294087] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.barbican_api_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.294259] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.barbican_endpoint = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.294448] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.barbican_endpoint_type = public {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.294620] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.barbican_region_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.294775] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.294932] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.295123] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.295299] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.295452] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.295613] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.number_of_retries = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.295773] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.retry_delay = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.295955] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.send_service_user_token = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.296131] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.296286] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.296438] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.verify_ssl = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.296588] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican.verify_ssl_path = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.296799] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.296972] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.auth_type = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.297144] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.297297] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.297455] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.297612] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.297761] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.297916] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.298075] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] barbican_service_user.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.298242] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.approle_role_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.298397] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.approle_secret_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.298549] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.298701] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.298857] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.299041] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.299208] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.299397] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.kv_mountpoint = secret {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.299572] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.kv_version = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.299732] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.namespace = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.299886] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.root_token_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.300053] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.300208] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.ssl_ca_crt_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.300358] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.300512] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.use_ssl = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.300674] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.300866] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.301166] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.301221] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.301362] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.connect_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.301512] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.connect_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.301662] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.endpoint_override = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.301817] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.301969] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.302129] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.max_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.302279] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.min_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.302431] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.region_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.302579] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.service_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.302744] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.service_type = identity {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.302900] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.303062] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.status_code_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.303224] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.status_code_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.303375] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.303547] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.valid_interfaces = ['internal', 'public'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.303704] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] keystone.version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.303926] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.connection_uri = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.304125] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.cpu_mode = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.304288] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.cpu_model_extra_flags = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.304450] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.cpu_models = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.304644] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.cpu_power_governor_high = performance {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.304804] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.cpu_power_governor_low = powersave {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.304957] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.cpu_power_management = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.305157] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.305329] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.device_detach_attempts = 8 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.305484] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.device_detach_timeout = 20 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.305643] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.disk_cachemodes = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.305797] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.disk_prefix = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.305971] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.enabled_perf_events = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.306138] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.file_backed_memory = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.306299] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.gid_maps = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.306449] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.hw_disk_discard = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.306607] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.hw_machine_type = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.306825] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.images_rbd_ceph_conf = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.306959] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.307146] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.307307] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.images_rbd_glance_store_name = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.307465] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.images_rbd_pool = rbd {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.307625] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.images_type = default {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.307775] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.images_volume_group = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.307929] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.inject_key = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.308093] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.inject_partition = -2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.308246] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.inject_password = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.308429] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.iscsi_iface = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.308585] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.iser_use_multipath = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.308740] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_bandwidth = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.308894] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_completion_timeout = 800 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.309060] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_downtime = 500 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.309217] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_downtime_delay = 75 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.309392] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_downtime_steps = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.309546] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_inbound_addr = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.309699] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_permit_auto_converge = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.309851] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_permit_post_copy = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.310008] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_scheme = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.310177] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_timeout_action = abort {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.310331] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_tunnelled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.310482] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_uri = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.310635] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.live_migration_with_native_tls = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.310786] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.max_queues = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.310964] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.mem_stats_period_seconds = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.311154] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.nfs_mount_options = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.311504] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.311680] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.num_aoe_discover_tries = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.311841] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.num_iser_scan_tries = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.311998] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.num_memory_encrypted_guests = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.312169] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.num_nvme_discover_tries = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.312330] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.num_pcie_ports = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.312508] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.num_volume_scan_tries = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.312719] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.pmem_namespaces = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.312875] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.quobyte_client_cfg = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.313129] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.313292] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rbd_connect_timeout = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.313446] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.313598] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.313747] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rbd_secret_uuid = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.313894] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rbd_user = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.314066] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.realtime_scheduler_priority = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.314236] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.remote_filesystem_transport = ssh {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.314389] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rescue_image_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.314537] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rescue_kernel_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.314687] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rescue_ramdisk_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.314842] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rng_dev_path = /dev/urandom {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.314995] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.rx_queue_size = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.315164] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.smbfs_mount_options = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.315370] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.315529] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.snapshot_compression = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.315678] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.snapshot_image_format = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.315884] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.316050] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.sparse_logical_volumes = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.316206] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.swtpm_enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.316364] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.swtpm_group = tss {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.316520] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.swtpm_user = tss {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.316693] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.sysinfo_serial = unique {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.316842] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.tx_queue_size = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.316995] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.uid_maps = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.317164] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.use_virtio_for_bridges = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.317322] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.virt_type = kvm {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.317477] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.volume_clear = zero {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.317627] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.volume_clear_size = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.317780] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.volume_use_multipath = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.317927] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.vzstorage_cache_path = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.318095] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.318250] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.vzstorage_mount_group = qemu {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.318408] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.vzstorage_mount_opts = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.318565] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.vzstorage_mount_perms = 0770 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.318763] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.318935] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.vzstorage_mount_user = stack {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.319098] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.319282] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.319447] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.auth_type = password {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.319596] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.319742] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.319894] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.320051] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.connect_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.320202] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.connect_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.320358] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.default_floating_pool = public {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.320504] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.endpoint_override = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.320679] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.extension_sync_interval = 600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.320834] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.http_retries = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.320987] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.321147] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.321296] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.max_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.321462] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.metadata_proxy_shared_secret = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.321609] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.min_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.321764] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.ovs_bridge = br-int {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.321918] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.physnets = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.322089] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.region_name = RegionOne {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.322262] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.service_metadata_proxy = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.322421] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.service_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.322581] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.service_type = network {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.322736] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.322887] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.status_code_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.323043] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.status_code_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.323193] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.323362] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.valid_interfaces = ['internal', 'public'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.323512] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] neutron.version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.323675] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] notifications.bdms_in_notifications = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.323847] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] notifications.default_level = INFO {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.324018] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] notifications.notification_format = unversioned {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.324178] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] notifications.notify_on_state_change = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.324349] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.324542] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] pci.alias = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.324733] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] pci.device_spec = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.324898] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] pci.report_in_placement = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.325093] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.325261] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.auth_type = password {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.325451] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.auth_url = http://10.180.1.21/identity {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.325605] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.325751] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.325906] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.326065] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.connect_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.326215] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.connect_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.326371] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.default_domain_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.326519] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.default_domain_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.326672] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.domain_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.326816] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.domain_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.326964] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.endpoint_override = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.327127] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.327274] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.327426] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.max_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.327574] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.min_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.327737] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.password = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.327888] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.project_domain_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.328058] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.project_domain_name = Default {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.328218] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.project_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.328382] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.project_name = service {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.328546] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.region_name = RegionOne {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.328695] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.service_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.328857] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.service_type = placement {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.329021] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.329173] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.status_code_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.329324] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.status_code_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.329474] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.system_scope = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.329623] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.329775] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.trust_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.329929] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.user_domain_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.330099] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.user_domain_name = Default {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.330253] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.user_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.330417] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.username = placement {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.330589] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.valid_interfaces = ['internal', 'public'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.330740] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] placement.version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.330912] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.cores = 20 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.331078] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.count_usage_from_placement = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.331240] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.driver = nova.quota.DbQuotaDriver {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.331420] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.injected_file_content_bytes = 10240 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.331652] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.injected_file_path_length = 255 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.331729] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.injected_files = 5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.331887] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.instances = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.332053] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.key_pairs = 100 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.332211] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.metadata_items = 128 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.332364] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.ram = 51200 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.332516] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.recheck_quota = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.332674] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.server_group_members = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.332833] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] quota.server_groups = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.332989] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rdp.enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.333301] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.333511] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.333703] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.333880] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.image_metadata_prefilter = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.334071] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.334252] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.max_attempts = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.334422] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.max_placement_results = 1000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.334598] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.334774] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.query_placement_for_availability_zone = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.334947] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.query_placement_for_image_type_support = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.335404] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.335404] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] scheduler.workers = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.335478] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.335644] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.335836] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.335999] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.336170] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.336324] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.336475] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.336690] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.336851] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.host_subset_size = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.337017] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.image_properties_default_architecture = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.337188] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.337343] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.isolated_hosts = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.337518] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.isolated_images = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.337674] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.max_instances_per_host = 50 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.337829] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.337986] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.pci_in_placement = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.338156] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.338311] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.338463] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.338616] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.338768] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.338919] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.339080] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.track_instance_changes = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.339251] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.339415] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] metrics.required = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.339569] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] metrics.weight_multiplier = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.339724] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] metrics.weight_of_unavailable = -10000.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.339881] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] metrics.weight_setting = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.340217] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.340385] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] serial_console.enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.340574] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] serial_console.port_range = 10000:20000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.340739] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.340902] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.341070] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] serial_console.serialproxy_port = 6083 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.341228] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.341399] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.auth_type = password {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.341549] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.341707] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.341851] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.342022] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.342167] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.342329] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.send_service_user_token = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.342512] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.342678] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] service_user.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.342846] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.agent_enabled = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.343032] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.343355] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.343574] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.html5proxy_host = 0.0.0.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.343740] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.html5proxy_port = 6082 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.343895] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.image_compression = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.344057] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.jpeg_compression = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.344212] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.playback_compression = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.344379] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.server_listen = 127.0.0.1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.344543] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.344694] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.streaming_mode = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.344843] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] spice.zlib_compression = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.345013] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] upgrade_levels.baseapi = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.345166] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] upgrade_levels.cert = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.345329] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] upgrade_levels.compute = auto {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.345481] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] upgrade_levels.conductor = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.345630] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] upgrade_levels.scheduler = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.345788] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.345949] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.auth_type = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.346112] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.346263] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.346417] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.346568] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.346722] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.346875] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.347032] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vendordata_dynamic_auth.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.347229] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.api_retry_count = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.347384] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.ca_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.347559] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.cache_prefix = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.347709] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.cluster_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.347864] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.connection_pool_size = 10 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.348015] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.console_delay_seconds = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.348166] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.datastore_regex = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.348317] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.host_ip = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.348464] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.host_password = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.348619] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.host_port = 443 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.348772] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.host_username = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.348927] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.349092] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.integration_bridge = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.349247] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.maximum_objects = 100 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.349401] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.pbm_default_policy = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.349557] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.pbm_enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.349854] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.pbm_wsdl_location = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.349891] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.350036] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.serial_port_proxy_uri = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.350190] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.serial_port_service_uri = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.350350] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.task_poll_interval = 0.5 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.350503] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.use_linked_clone = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.350662] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.vnc_keymap = en-us {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.350818] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.vnc_port = 5900 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.350971] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vmware.vnc_port_total = 10000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.351185] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.auth_schemes = ['none'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.351443] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.enabled = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.351664] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.351891] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.novncproxy_host = 0.0.0.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.352011] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.novncproxy_port = 6080 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.352189] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.server_listen = 127.0.0.1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.352356] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.352511] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.vencrypt_ca_certs = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.352663] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.vencrypt_client_cert = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.352816] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] vnc.vencrypt_client_key = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.353057] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.353189] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.disable_fallback_pcpu_query = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.353345] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.disable_group_policy_check_upcall = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.353499] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.353654] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.disable_rootwrap = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.353811] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.enable_numa_live_migration = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.353964] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.354129] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.354284] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.handle_virt_lifecycle_events = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.354441] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.libvirt_disable_apic = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.354592] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.never_download_image_if_on_rbd = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.354755] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.354904] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.355071] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.355226] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.355380] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.355533] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.355692] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.355846] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.355999] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.356188] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.356369] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.client_socket_timeout = 900 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.356531] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.default_pool_size = 1000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.356697] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.keep_alive = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.356849] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.max_header_line = 16384 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.357007] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.secure_proxy_ssl_header = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.357166] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.ssl_ca_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.357320] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.ssl_cert_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.357470] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.ssl_key_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.357626] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.tcp_keepidle = 600 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.357794] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.357956] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] zvm.ca_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.358117] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] zvm.cloud_connector_url = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.358337] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.358505] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] zvm.reachable_timeout = 300 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.358727] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.enforce_new_defaults = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.358898] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.enforce_scope = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.359289] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.policy_default_rule = default {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.359289] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.359467] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.policy_file = policy.yaml {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.359659] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.359831] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.359986] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.360168] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.360325] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.360513] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.360690] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.360897] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.connection_string = messaging:// {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.361095] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.enabled = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.361275] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.es_doc_type = notification {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.361448] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.es_scroll_size = 10000 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.361612] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.es_scroll_time = 2m {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.361766] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.filter_error_trace = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.361932] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.hmac_keys = SECRET_KEY {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.362099] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.sentinel_service_name = mymaster {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.362285] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.socket_timeout = 0.1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.362444] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] profiler.trace_sqlalchemy = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.362637] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] remote_debug.host = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.362806] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] remote_debug.port = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.362994] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.363172] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.363333] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.363487] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.363648] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.363806] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.363966] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.364138] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.364293] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.364441] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.364604] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.364780] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.364944] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.365115] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.365268] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.365437] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.365622] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.365746] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.365923] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.366096] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.366253] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.366412] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.366566] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.366746] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.366902] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.367074] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.ssl = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.367238] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.367399] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.367555] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.367751] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.367882] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_rabbit.ssl_version = {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.368093] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.368258] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_notifications.retry = -1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.368436] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.368603] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_messaging_notifications.transport_url = **** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.368797] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.auth_section = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.368955] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.auth_type = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.369131] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.cafile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.369281] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.certfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.369434] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.collect_timing = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.369583] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.connect_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.369732] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.connect_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.369900] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.endpoint_id = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.370064] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.endpoint_override = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.370215] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.insecure = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.370361] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.keyfile = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.370508] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.max_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.370652] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.min_version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.370804] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.region_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.370969] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.service_name = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.371128] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.service_type = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.371279] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.split_loggers = False {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.371425] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.status_code_retries = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.371577] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.status_code_retry_delay = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.371721] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.timeout = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.371871] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.valid_interfaces = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.372026] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_limit.version = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.372224] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_reports.file_event_handler = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.372381] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.372529] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] oslo_reports.log_dir = None {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} [ 463.372654] nova-conductor[52380]: DEBUG oslo_service.service [None req-5016ad94-cdaf-4683-aa68-5719f363d419 None None] ******************************************************************************** {{(pid=52380) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}} [ 549.406993] nova-conductor[52800]: DEBUG oslo_db.sqlalchemy.engines [None req-897f61cb-608b-4a12-841b-d39b375bee0e None None] Parent process 52380 forked (52800) with an open database connection, which is being discarded and recreated. {{(pid=52800) checkout /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:434}} [ 592.449577] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Took 0.64 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 592.485751] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.486129] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.487814] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.494886] nova-conductor[52800]: DEBUG oslo_db.sqlalchemy.engines [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52800) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 592.569412] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.570222] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.570222] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.570527] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.570743] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.571063] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.582397] nova-conductor[52800]: DEBUG oslo_db.sqlalchemy.engines [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52800) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 592.613034] nova-conductor[52800]: DEBUG nova.quota [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Getting quotas for project 62b2e3dba50f46108124ecd6f560df03. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 592.617862] nova-conductor[52800]: DEBUG nova.quota [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Getting quotas for user 8116a4b14c4148ce9863afdb6dd7d571 and project 62b2e3dba50f46108124ecd6f560df03. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 592.633967] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] [instance: f1894e2a-156c-420c-91af-a4eedaafb017] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 592.633967] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.633967] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.633967] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.642704] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] [instance: f1894e2a-156c-420c-91af-a4eedaafb017] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 592.643464] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.643659] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.643844] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.690363] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.690612] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.690779] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.691088] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=52800) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 592.691231] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Acquired lock "compute-rpcapi-router" {{(pid=52800) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 592.691767] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.691953] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.693581] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.693581] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 592.693581] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 592.693835] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 592.704137] nova-conductor[52800]: INFO nova.compute.rpcapi [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Automatically selected compute RPC version 6.2 from minimum service version 66 [ 592.704623] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-40b15e4b-9d45-4191-a8a5-cfafa0e00adb None None] Releasing lock "compute-rpcapi-router" {{(pid=52800) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 593.068951] nova-conductor[52799]: DEBUG oslo_db.sqlalchemy.engines [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Parent process 52380 forked (52799) with an open database connection, which is being discarded and recreated. {{(pid=52799) checkout /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:434}} [ 593.353446] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Took 0.27 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 593.375736] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.376098] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.377963] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.383162] nova-conductor[52799]: DEBUG oslo_db.sqlalchemy.engines [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52799) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 593.443451] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.444281] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.444281] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.444515] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.444692] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.444842] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.455592] nova-conductor[52799]: DEBUG oslo_db.sqlalchemy.engines [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=52799) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} [ 593.470661] nova-conductor[52799]: DEBUG nova.quota [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Getting quotas for project 73f02488e0f04572afd9b672c2b5cf0f. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 593.472881] nova-conductor[52799]: DEBUG nova.quota [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Getting quotas for user 74c20d44ea7e4427ae91b7d5cb67941f and project 73f02488e0f04572afd9b672c2b5cf0f. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 593.479573] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] [instance: c731bc6a-9b0d-4e3a-b5ca-009d79896d27] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 593.479573] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.479797] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.479866] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.483929] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] [instance: c731bc6a-9b0d-4e3a-b5ca-009d79896d27] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 593.484626] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.484868] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.484977] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.514764] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.516997] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.516997] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.516997] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=52799) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 593.516997] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Acquired lock "compute-rpcapi-router" {{(pid=52799) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 593.517260] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.517260] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.517260] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.517260] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.517260] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.517412] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 593.523828] nova-conductor[52799]: INFO nova.compute.rpcapi [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Automatically selected compute RPC version 6.2 from minimum service version 66 [ 593.524240] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7921ccdf-ea0e-42e5-b913-384f1030739d None None] Releasing lock "compute-rpcapi-router" {{(pid=52799) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 593.961020] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 593.986673] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 593.986894] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 593.987070] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 594.071871] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 594.073675] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 594.073675] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 594.073675] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 594.073675] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 594.073919] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 594.083203] nova-conductor[52800]: DEBUG nova.quota [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Getting quotas for project 859a3caa90ab40879a27f7eb2ba8908b. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 594.085579] nova-conductor[52800]: DEBUG nova.quota [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Getting quotas for user 1e7b888f72654a79b43f8a2c47299416 and project 859a3caa90ab40879a27f7eb2ba8908b. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 594.092637] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 09e936da-040a-438a-a320-28616de7bb75] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 594.093403] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 594.093626] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 594.093802] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 594.097795] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 09e936da-040a-438a-a320-28616de7bb75] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 594.097795] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 594.097963] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 594.098089] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 594.116662] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 594.116877] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 594.117062] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.017098] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 596.039991] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.039991] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.040116] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.073894] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 596.079277] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.079564] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.083018] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.083018] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.083018] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.083018] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.085314] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.085528] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.086068] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.091669] nova-conductor[52799]: DEBUG nova.quota [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Getting quotas for project bd26fa229f324ee7aca2569d83ed0032. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 596.093450] nova-conductor[52799]: DEBUG nova.quota [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Getting quotas for user aedbb3047ca14d2aa7cd02b30a892ba2 and project bd26fa229f324ee7aca2569d83ed0032. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 596.105486] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] [instance: 3681b35b-c962-4e80-8f9c-df0db2f515e9] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 596.105486] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.105486] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.105486] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.107347] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] [instance: 3681b35b-c962-4e80-8f9c-df0db2f515e9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 596.108156] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.108471] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.108705] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.133641] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.135409] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.135601] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.135982] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.136189] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.136373] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.147509] nova-conductor[52800]: DEBUG nova.quota [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Getting quotas for project fc4180d54c1c427081e8004cc159478f. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 596.150999] nova-conductor[52800]: DEBUG nova.quota [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Getting quotas for user a30fc787cf3745b9bc8e385a9f26749f and project fc4180d54c1c427081e8004cc159478f. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 596.156740] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.156973] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.157047] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: dbb50c25-381f-4878-945b-170f2681f2ae] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 596.157152] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.157482] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.160219] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.160219] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.162814] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: dbb50c25-381f-4878-945b-170f2681f2ae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 596.163748] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.163748] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.163903] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 596.181913] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 596.182162] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 596.182333] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 598.093049] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 598.107284] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 598.107510] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 598.107737] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 598.143873] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 598.144343] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 598.144343] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 598.144640] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 598.144846] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 598.145024] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 598.162303] nova-conductor[52799]: DEBUG nova.quota [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Getting quotas for project 33983ea7f05b435da567e01fa0715162. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 598.168287] nova-conductor[52799]: DEBUG nova.quota [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Getting quotas for user be9e4c9d8e1249f7b4691aa88666769d and project 33983ea7f05b435da567e01fa0715162. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 598.178755] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 4dbfaeea-229a-4ed1-afb2-bd8e167a1385] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 598.179383] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 598.179669] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 598.179799] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 598.186715] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 4dbfaeea-229a-4ed1-afb2-bd8e167a1385] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 598.187257] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 598.187257] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 598.187401] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 598.206547] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 598.206663] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 598.206870] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 600.727293] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 600.738853] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 600.739095] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 600.739268] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 600.764235] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 600.764459] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 600.764629] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 600.764970] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 600.765167] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 600.765325] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 600.774808] nova-conductor[52800]: DEBUG nova.quota [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Getting quotas for project 5275c625e8af40b28d4612c1473115df. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 600.777330] nova-conductor[52800]: DEBUG nova.quota [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Getting quotas for user 8c45812d2d1b498eabb8228cdd1fd48b and project 5275c625e8af40b28d4612c1473115df. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 600.786258] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] [instance: 93b26227-ad64-4343-aed9-ba6622aaf83e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 600.786751] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 600.786951] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 600.787135] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 600.795635] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] [instance: 93b26227-ad64-4343-aed9-ba6622aaf83e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 600.796452] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 600.796452] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 600.796576] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 600.810863] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 600.811053] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 600.811227] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 607.353084] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] [instance: f1894e2a-156c-420c-91af-a4eedaafb017] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 61b1b63d-b5cb-4d06-b6ff-cda84a011f31, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance f1894e2a-156c-420c-91af-a4eedaafb017 was re-scheduled: Binding failed for port 61b1b63d-b5cb-4d06-b6ff-cda84a011f31, please check neutron logs for more information.\n'] [ 607.354206] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 607.354572] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1894e2a-156c-420c-91af-a4eedaafb017.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1894e2a-156c-420c-91af-a4eedaafb017. [ 607.355076] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] [instance: f1894e2a-156c-420c-91af-a4eedaafb017] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance f1894e2a-156c-420c-91af-a4eedaafb017. [ 607.427796] nova-conductor[52799]: DEBUG nova.network.neutron [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] [instance: f1894e2a-156c-420c-91af-a4eedaafb017] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 607.574281] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] [instance: c731bc6a-9b0d-4e3a-b5ca-009d79896d27] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port bd6c3933-bc0e-43ec-b6e3-dae79283705c, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c731bc6a-9b0d-4e3a-b5ca-009d79896d27 was re-scheduled: Binding failed for port bd6c3933-bc0e-43ec-b6e3-dae79283705c, please check neutron logs for more information.\n'] [ 607.576087] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 607.576087] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c731bc6a-9b0d-4e3a-b5ca-009d79896d27.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c731bc6a-9b0d-4e3a-b5ca-009d79896d27. [ 607.576087] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] [instance: c731bc6a-9b0d-4e3a-b5ca-009d79896d27] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c731bc6a-9b0d-4e3a-b5ca-009d79896d27. [ 607.608523] nova-conductor[52799]: DEBUG nova.network.neutron [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] [instance: c731bc6a-9b0d-4e3a-b5ca-009d79896d27] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 607.957322] nova-conductor[52799]: DEBUG nova.network.neutron [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] [instance: c731bc6a-9b0d-4e3a-b5ca-009d79896d27] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 607.967340] nova-conductor[52799]: DEBUG nova.network.neutron [None req-6263ba31-4ac7-4ed2-9e75-ff30010bd9d2 tempest-ServersAdminNegativeTestJSON-2079946617 tempest-ServersAdminNegativeTestJSON-2079946617-project-member] [instance: c731bc6a-9b0d-4e3a-b5ca-009d79896d27] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 608.251021] nova-conductor[52799]: DEBUG nova.network.neutron [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] [instance: f1894e2a-156c-420c-91af-a4eedaafb017] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 608.260024] nova-conductor[52799]: DEBUG nova.network.neutron [None req-d2ce4208-2032-4f1b-8d6a-890bf3f1fa09 tempest-ServerDiagnosticsTest-844409055 tempest-ServerDiagnosticsTest-844409055-project-member] [instance: f1894e2a-156c-420c-91af-a4eedaafb017] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 608.383791] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 09e936da-040a-438a-a320-28616de7bb75] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4aa0c292-e1ce-4f56-9399-62866cbc7ba7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 09e936da-040a-438a-a320-28616de7bb75 was re-scheduled: Binding failed for port 4aa0c292-e1ce-4f56-9399-62866cbc7ba7, please check neutron logs for more information.\n'] [ 608.384818] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 608.384818] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 09e936da-040a-438a-a320-28616de7bb75.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 09e936da-040a-438a-a320-28616de7bb75. [ 608.385018] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 09e936da-040a-438a-a320-28616de7bb75] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 09e936da-040a-438a-a320-28616de7bb75. [ 608.418808] nova-conductor[52799]: DEBUG nova.network.neutron [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 09e936da-040a-438a-a320-28616de7bb75] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 608.537733] nova-conductor[52799]: DEBUG nova.network.neutron [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 09e936da-040a-438a-a320-28616de7bb75] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 608.549191] nova-conductor[52799]: DEBUG nova.network.neutron [None req-797add29-9b3a-43bc-9bf6-f89dc30b11cd tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 09e936da-040a-438a-a320-28616de7bb75] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 611.001927] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] [instance: 3681b35b-c962-4e80-8f9c-df0db2f515e9] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 83c6510e-dc8c-4f57-b4d1-a3af8164ac57, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 3681b35b-c962-4e80-8f9c-df0db2f515e9 was re-scheduled: Binding failed for port 83c6510e-dc8c-4f57-b4d1-a3af8164ac57, please check neutron logs for more information.\n'] [ 611.001927] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 611.001927] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3681b35b-c962-4e80-8f9c-df0db2f515e9.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3681b35b-c962-4e80-8f9c-df0db2f515e9. [ 611.001927] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] [instance: 3681b35b-c962-4e80-8f9c-df0db2f515e9] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 3681b35b-c962-4e80-8f9c-df0db2f515e9. [ 611.043668] nova-conductor[52799]: DEBUG nova.network.neutron [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] [instance: 3681b35b-c962-4e80-8f9c-df0db2f515e9] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 611.180640] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: dbb50c25-381f-4878-945b-170f2681f2ae] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 8d00bd2a-2402-4f54-9624-959f29f21eda, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance dbb50c25-381f-4878-945b-170f2681f2ae was re-scheduled: Binding failed for port 8d00bd2a-2402-4f54-9624-959f29f21eda, please check neutron logs for more information.\n'] [ 611.184189] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 611.184189] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dbb50c25-381f-4878-945b-170f2681f2ae.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dbb50c25-381f-4878-945b-170f2681f2ae. [ 611.184189] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: dbb50c25-381f-4878-945b-170f2681f2ae] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance dbb50c25-381f-4878-945b-170f2681f2ae. [ 611.211837] nova-conductor[52799]: DEBUG nova.network.neutron [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: dbb50c25-381f-4878-945b-170f2681f2ae] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 611.225210] nova-conductor[52799]: DEBUG nova.network.neutron [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] [instance: 3681b35b-c962-4e80-8f9c-df0db2f515e9] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 611.228806] nova-conductor[52799]: DEBUG nova.network.neutron [None req-f5c1b416-d9aa-42f4-ad1c-7382d98aa4e0 tempest-TenantUsagesTestJSON-2036990172 tempest-TenantUsagesTestJSON-2036990172-project-member] [instance: 3681b35b-c962-4e80-8f9c-df0db2f515e9] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 611.535649] nova-conductor[52799]: DEBUG nova.network.neutron [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: dbb50c25-381f-4878-945b-170f2681f2ae] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 611.546948] nova-conductor[52799]: DEBUG nova.network.neutron [None req-733d259c-18d3-48c3-94b3-b8409db5e48a tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: dbb50c25-381f-4878-945b-170f2681f2ae] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 613.241906] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 4dbfaeea-229a-4ed1-afb2-bd8e167a1385] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 25193a61-bed4-4d25-8d37-d6270569cf1a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4dbfaeea-229a-4ed1-afb2-bd8e167a1385 was re-scheduled: Binding failed for port 25193a61-bed4-4d25-8d37-d6270569cf1a, please check neutron logs for more information.\n'] [ 613.244644] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 613.244644] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4dbfaeea-229a-4ed1-afb2-bd8e167a1385.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4dbfaeea-229a-4ed1-afb2-bd8e167a1385. [ 613.244644] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 4dbfaeea-229a-4ed1-afb2-bd8e167a1385] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4dbfaeea-229a-4ed1-afb2-bd8e167a1385. [ 613.271021] nova-conductor[52799]: DEBUG nova.network.neutron [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 4dbfaeea-229a-4ed1-afb2-bd8e167a1385] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 613.371142] nova-conductor[52799]: DEBUG nova.network.neutron [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 4dbfaeea-229a-4ed1-afb2-bd8e167a1385] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 613.377221] nova-conductor[52799]: DEBUG nova.network.neutron [None req-97f64e23-a856-4e36-b538-6171155631ac tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 4dbfaeea-229a-4ed1-afb2-bd8e167a1385] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 615.442855] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] [instance: 93b26227-ad64-4343-aed9-ba6622aaf83e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 26abace3-118f-40c6-882e-a81b73e76917, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 93b26227-ad64-4343-aed9-ba6622aaf83e was re-scheduled: Binding failed for port 26abace3-118f-40c6-882e-a81b73e76917, please check neutron logs for more information.\n'] [ 615.443870] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 615.444174] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93b26227-ad64-4343-aed9-ba6622aaf83e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93b26227-ad64-4343-aed9-ba6622aaf83e. [ 615.444579] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] [instance: 93b26227-ad64-4343-aed9-ba6622aaf83e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 93b26227-ad64-4343-aed9-ba6622aaf83e. [ 615.507914] nova-conductor[52800]: DEBUG nova.network.neutron [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] [instance: 93b26227-ad64-4343-aed9-ba6622aaf83e] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 615.851946] nova-conductor[52800]: DEBUG nova.network.neutron [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] [instance: 93b26227-ad64-4343-aed9-ba6622aaf83e] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 615.858179] nova-conductor[52800]: DEBUG nova.network.neutron [None req-b2a35850-14c6-4f74-8a9b-fc0caf713225 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501 tempest-FloatingIPsAssociationNegativeTestJSON-1043172501-project-member] [instance: 93b26227-ad64-4343-aed9-ba6622aaf83e] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 616.458994] nova-conductor[52800]: Traceback (most recent call last): [ 616.458994] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 616.458994] nova-conductor[52800]: return func(*args, **kwargs) [ 616.458994] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 616.458994] nova-conductor[52800]: selections = self._select_destinations( [ 616.458994] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 616.458994] nova-conductor[52800]: selections = self._schedule( [ 616.458994] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 616.458994] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 616.458994] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 616.458994] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 616.458994] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.458994] nova-conductor[52800]: ERROR nova.conductor.manager [ 616.470982] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 616.471265] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 616.471460] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 616.536563] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 4460c09b-b99a-4786-99e5-01b88b7157c4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 616.537637] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 616.537892] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 616.538160] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 616.545433] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 616.545433] nova-conductor[52800]: Traceback (most recent call last): [ 616.545433] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 616.545433] nova-conductor[52800]: return func(*args, **kwargs) [ 616.545433] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 616.545433] nova-conductor[52800]: selections = self._select_destinations( [ 616.545433] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 616.545433] nova-conductor[52800]: selections = self._schedule( [ 616.545433] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 616.545433] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 616.545433] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 616.545433] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 616.545433] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 616.545433] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 616.546743] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-9a7ddac2-405b-4ab4-80f1-02f586d1d26a tempest-DeleteServersAdminTestJSON-306082810 tempest-DeleteServersAdminTestJSON-306082810-project-member] [instance: 4460c09b-b99a-4786-99e5-01b88b7157c4] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 618.006692] nova-conductor[52799]: Traceback (most recent call last): [ 618.006692] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 618.006692] nova-conductor[52799]: return func(*args, **kwargs) [ 618.006692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 618.006692] nova-conductor[52799]: selections = self._select_destinations( [ 618.006692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 618.006692] nova-conductor[52799]: selections = self._schedule( [ 618.006692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 618.006692] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 618.006692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 618.006692] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 618.006692] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.006692] nova-conductor[52799]: ERROR nova.conductor.manager [ 618.021722] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 618.021947] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 618.022126] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 618.101886] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] [instance: 381974bf-c879-46e5-8d96-495a9b9eb982] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 618.102619] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 618.102858] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 618.103037] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 618.112675] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 618.112675] nova-conductor[52799]: Traceback (most recent call last): [ 618.112675] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 618.112675] nova-conductor[52799]: return func(*args, **kwargs) [ 618.112675] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 618.112675] nova-conductor[52799]: selections = self._select_destinations( [ 618.112675] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 618.112675] nova-conductor[52799]: selections = self._schedule( [ 618.112675] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 618.112675] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 618.112675] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 618.112675] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 618.112675] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 618.112675] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 618.113889] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-09cfe728-e16e-44b9-8f3e-e56882817815 tempest-ServersAdmin275Test-1865677316 tempest-ServersAdmin275Test-1865677316-project-member] [instance: 381974bf-c879-46e5-8d96-495a9b9eb982] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.043717] nova-conductor[52800]: Traceback (most recent call last): [ 619.043717] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 619.043717] nova-conductor[52800]: return func(*args, **kwargs) [ 619.043717] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 619.043717] nova-conductor[52800]: selections = self._select_destinations( [ 619.043717] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 619.043717] nova-conductor[52800]: selections = self._schedule( [ 619.043717] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 619.043717] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 619.043717] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 619.043717] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 619.043717] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.043717] nova-conductor[52800]: ERROR nova.conductor.manager [ 619.054167] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.054392] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.054561] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.107659] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] [instance: bf3febf6-cd6b-4c50-85b1-537b541a7d62] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 619.108388] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.108609] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.108809] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.113239] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 619.113239] nova-conductor[52800]: Traceback (most recent call last): [ 619.113239] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 619.113239] nova-conductor[52800]: return func(*args, **kwargs) [ 619.113239] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 619.113239] nova-conductor[52800]: selections = self._select_destinations( [ 619.113239] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 619.113239] nova-conductor[52800]: selections = self._schedule( [ 619.113239] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 619.113239] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 619.113239] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 619.113239] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 619.113239] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 619.113239] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.113908] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-efa863cf-3e03-4569-8f37-c34e8e3ffdf1 tempest-ServerDiagnosticsNegativeTest-578356840 tempest-ServerDiagnosticsNegativeTest-578356840-project-member] [instance: bf3febf6-cd6b-4c50-85b1-537b541a7d62] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.671514] nova-conductor[52799]: Traceback (most recent call last): [ 619.671514] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 619.671514] nova-conductor[52799]: return func(*args, **kwargs) [ 619.671514] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 619.671514] nova-conductor[52799]: selections = self._select_destinations( [ 619.671514] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 619.671514] nova-conductor[52799]: selections = self._schedule( [ 619.671514] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 619.671514] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 619.671514] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 619.671514] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 619.671514] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.671514] nova-conductor[52799]: ERROR nova.conductor.manager [ 619.682795] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.683057] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.683257] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.742134] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] [instance: 0f5806b0-1b69-404b-861f-cd98d05639f2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 619.743236] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 619.743565] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 619.743844] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 619.749606] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 619.749606] nova-conductor[52799]: Traceback (most recent call last): [ 619.749606] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 619.749606] nova-conductor[52799]: return func(*args, **kwargs) [ 619.749606] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 619.749606] nova-conductor[52799]: selections = self._select_destinations( [ 619.749606] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 619.749606] nova-conductor[52799]: selections = self._schedule( [ 619.749606] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 619.749606] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 619.749606] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 619.749606] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 619.749606] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 619.749606] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 619.753527] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-06cb6bb5-7e21-4985-b6da-c5656b3ba35e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] [instance: 0f5806b0-1b69-404b-861f-cd98d05639f2] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 620.692952] nova-conductor[52800]: Traceback (most recent call last): [ 620.692952] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 620.692952] nova-conductor[52800]: return func(*args, **kwargs) [ 620.692952] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 620.692952] nova-conductor[52800]: selections = self._select_destinations( [ 620.692952] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 620.692952] nova-conductor[52800]: selections = self._schedule( [ 620.692952] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 620.692952] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 620.692952] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 620.692952] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 620.692952] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.692952] nova-conductor[52800]: ERROR nova.conductor.manager [ 620.699875] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.700151] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.700327] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.755359] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] [instance: 6ee76f18-9840-4d27-bb7c-94571dc7d651] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 620.756853] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 620.756853] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 620.756853] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 620.759896] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 620.759896] nova-conductor[52800]: Traceback (most recent call last): [ 620.759896] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 620.759896] nova-conductor[52800]: return func(*args, **kwargs) [ 620.759896] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 620.759896] nova-conductor[52800]: selections = self._select_destinations( [ 620.759896] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 620.759896] nova-conductor[52800]: selections = self._schedule( [ 620.759896] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 620.759896] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 620.759896] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 620.759896] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 620.759896] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 620.759896] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 620.761046] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-44702fa8-6484-4dbd-b384-0bec581caf2a tempest-ServersWithSpecificFlavorTestJSON-430937670 tempest-ServersWithSpecificFlavorTestJSON-430937670-project-member] [instance: 6ee76f18-9840-4d27-bb7c-94571dc7d651] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 621.201531] nova-conductor[52799]: Traceback (most recent call last): [ 621.201531] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 621.201531] nova-conductor[52799]: return func(*args, **kwargs) [ 621.201531] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 621.201531] nova-conductor[52799]: selections = self._select_destinations( [ 621.201531] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 621.201531] nova-conductor[52799]: selections = self._schedule( [ 621.201531] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 621.201531] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 621.201531] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 621.201531] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 621.201531] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.201531] nova-conductor[52799]: ERROR nova.conductor.manager [ 621.210067] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 621.210067] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 621.210212] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 621.259723] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] [instance: 86c1eb22-5da1-4d79-8db6-b131e0a0d1c3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 621.260467] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 621.260679] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 621.260847] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 621.264044] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 621.264044] nova-conductor[52799]: Traceback (most recent call last): [ 621.264044] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 621.264044] nova-conductor[52799]: return func(*args, **kwargs) [ 621.264044] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 621.264044] nova-conductor[52799]: selections = self._select_destinations( [ 621.264044] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 621.264044] nova-conductor[52799]: selections = self._schedule( [ 621.264044] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 621.264044] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 621.264044] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 621.264044] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 621.264044] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 621.264044] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 621.264699] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d68d1347-88bd-4d90-9505-4b36b46d1fc8 tempest-ImagesNegativeTestJSON-295564784 tempest-ImagesNegativeTestJSON-295564784-project-member] [instance: 86c1eb22-5da1-4d79-8db6-b131e0a0d1c3] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 624.158705] nova-conductor[52800]: Traceback (most recent call last): [ 624.158705] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 624.158705] nova-conductor[52800]: return func(*args, **kwargs) [ 624.158705] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 624.158705] nova-conductor[52800]: selections = self._select_destinations( [ 624.158705] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 624.158705] nova-conductor[52800]: selections = self._schedule( [ 624.158705] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 624.158705] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 624.158705] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 624.158705] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 624.158705] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.158705] nova-conductor[52800]: ERROR nova.conductor.manager [ 624.170345] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.170725] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.171036] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.231853] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] [instance: 6ba8cf5b-f60b-462b-a955-eaeba2849c0d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 624.232662] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 624.232877] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 624.233059] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 624.236400] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 624.236400] nova-conductor[52800]: Traceback (most recent call last): [ 624.236400] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 624.236400] nova-conductor[52800]: return func(*args, **kwargs) [ 624.236400] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 624.236400] nova-conductor[52800]: selections = self._select_destinations( [ 624.236400] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 624.236400] nova-conductor[52800]: selections = self._schedule( [ 624.236400] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 624.236400] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 624.236400] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 624.236400] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 624.236400] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 624.236400] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 624.236966] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-556dd900-feb1-4925-8ffd-74662c06ddbc tempest-ServersTestFqdnHostnames-46354780 tempest-ServersTestFqdnHostnames-46354780-project-member] [instance: 6ba8cf5b-f60b-462b-a955-eaeba2849c0d] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 625.117703] nova-conductor[52799]: Traceback (most recent call last): [ 625.117703] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 625.117703] nova-conductor[52799]: return func(*args, **kwargs) [ 625.117703] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 625.117703] nova-conductor[52799]: selections = self._select_destinations( [ 625.117703] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 625.117703] nova-conductor[52799]: selections = self._schedule( [ 625.117703] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 625.117703] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 625.117703] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 625.117703] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 625.117703] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.117703] nova-conductor[52799]: ERROR nova.conductor.manager [ 625.124649] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 625.124953] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 625.126083] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 625.177133] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] [instance: 0f32ff9f-758b-4616-9291-f141b74acdf7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 625.178366] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 625.178700] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 625.178950] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 625.182611] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 625.182611] nova-conductor[52799]: Traceback (most recent call last): [ 625.182611] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 625.182611] nova-conductor[52799]: return func(*args, **kwargs) [ 625.182611] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 625.182611] nova-conductor[52799]: selections = self._select_destinations( [ 625.182611] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 625.182611] nova-conductor[52799]: selections = self._schedule( [ 625.182611] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 625.182611] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 625.182611] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 625.182611] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 625.182611] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 625.182611] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 625.183863] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-4774e3e4-1696-4a30-b5df-b0f9fef76e4f tempest-VolumesAssistedSnapshotsTest-1513480278 tempest-VolumesAssistedSnapshotsTest-1513480278-project-member] [instance: 0f32ff9f-758b-4616-9291-f141b74acdf7] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 626.459839] nova-conductor[52800]: Traceback (most recent call last): [ 626.459839] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 626.459839] nova-conductor[52800]: return func(*args, **kwargs) [ 626.459839] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 626.459839] nova-conductor[52800]: selections = self._select_destinations( [ 626.459839] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 626.459839] nova-conductor[52800]: selections = self._schedule( [ 626.459839] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 626.459839] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 626.459839] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 626.459839] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 626.459839] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.459839] nova-conductor[52800]: ERROR nova.conductor.manager [ 626.469372] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.469612] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.469799] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.519836] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: d80cb5ce-2aaa-478d-85fa-23eabb8c9ad8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 626.520688] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 626.521050] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 626.521141] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 626.524455] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 626.524455] nova-conductor[52800]: Traceback (most recent call last): [ 626.524455] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 626.524455] nova-conductor[52800]: return func(*args, **kwargs) [ 626.524455] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 626.524455] nova-conductor[52800]: selections = self._select_destinations( [ 626.524455] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 626.524455] nova-conductor[52800]: selections = self._schedule( [ 626.524455] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 626.524455] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 626.524455] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 626.524455] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 626.524455] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 626.524455] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 626.525077] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-63528711-7e54-445e-8353-14eab703533e tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: d80cb5ce-2aaa-478d-85fa-23eabb8c9ad8] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 627.257660] nova-conductor[52799]: Traceback (most recent call last): [ 627.257660] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 627.257660] nova-conductor[52799]: return func(*args, **kwargs) [ 627.257660] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 627.257660] nova-conductor[52799]: selections = self._select_destinations( [ 627.257660] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 627.257660] nova-conductor[52799]: selections = self._schedule( [ 627.257660] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 627.257660] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 627.257660] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 627.257660] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 627.257660] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.257660] nova-conductor[52799]: ERROR nova.conductor.manager [ 627.267425] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 627.267644] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 627.267809] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 627.341886] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] [instance: a514e44b-9214-4a83-aba0-4af32e95c94c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 627.342608] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 627.342811] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 627.342974] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 627.346434] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 627.346434] nova-conductor[52799]: Traceback (most recent call last): [ 627.346434] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 627.346434] nova-conductor[52799]: return func(*args, **kwargs) [ 627.346434] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 627.346434] nova-conductor[52799]: selections = self._select_destinations( [ 627.346434] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 627.346434] nova-conductor[52799]: selections = self._schedule( [ 627.346434] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 627.346434] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 627.346434] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 627.346434] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 627.346434] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 627.346434] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 627.347075] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-68056aff-2fee-47c8-8ef4-2644a4005bb6 tempest-ServerExternalEventsTest-624411406 tempest-ServerExternalEventsTest-624411406-project-member] [instance: a514e44b-9214-4a83-aba0-4af32e95c94c] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 628.995457] nova-conductor[52800]: Traceback (most recent call last): [ 628.995457] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 628.995457] nova-conductor[52800]: return func(*args, **kwargs) [ 628.995457] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 628.995457] nova-conductor[52800]: selections = self._select_destinations( [ 628.995457] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 628.995457] nova-conductor[52800]: selections = self._schedule( [ 628.995457] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 628.995457] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 628.995457] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 628.995457] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 628.995457] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 628.995457] nova-conductor[52800]: ERROR nova.conductor.manager [ 629.002982] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 629.003509] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 629.003950] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 629.070763] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] [instance: 7b3fab69-c129-4abe-aa5a-1f873260df19] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 629.071721] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 629.073038] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 629.073038] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 629.076814] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 629.076814] nova-conductor[52800]: Traceback (most recent call last): [ 629.076814] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 629.076814] nova-conductor[52800]: return func(*args, **kwargs) [ 629.076814] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 629.076814] nova-conductor[52800]: selections = self._select_destinations( [ 629.076814] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 629.076814] nova-conductor[52800]: selections = self._schedule( [ 629.076814] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 629.076814] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 629.076814] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 629.076814] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 629.076814] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 629.076814] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 629.077358] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-cb8594b1-2984-48bb-b2a1-c1d0972a89d4 tempest-FloatingIPsAssociationTestJSON-1610426689 tempest-FloatingIPsAssociationTestJSON-1610426689-project-member] [instance: 7b3fab69-c129-4abe-aa5a-1f873260df19] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 636.581232] nova-conductor[52799]: Traceback (most recent call last): [ 636.581232] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 636.581232] nova-conductor[52799]: return func(*args, **kwargs) [ 636.581232] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 636.581232] nova-conductor[52799]: selections = self._select_destinations( [ 636.581232] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 636.581232] nova-conductor[52799]: selections = self._schedule( [ 636.581232] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 636.581232] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 636.581232] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 636.581232] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 636.581232] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.581232] nova-conductor[52799]: ERROR nova.conductor.manager [ 636.593474] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 636.593474] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 636.593474] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 636.644987] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] [instance: eeae9a99-e184-49ea-880f-0a429d893129] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 636.645841] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 636.646108] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 636.646323] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 636.649635] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 636.649635] nova-conductor[52799]: Traceback (most recent call last): [ 636.649635] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 636.649635] nova-conductor[52799]: return func(*args, **kwargs) [ 636.649635] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 636.649635] nova-conductor[52799]: selections = self._select_destinations( [ 636.649635] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 636.649635] nova-conductor[52799]: selections = self._schedule( [ 636.649635] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 636.649635] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 636.649635] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 636.649635] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 636.649635] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 636.649635] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 636.650656] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d3f4ff63-8cf9-487f-a6a2-aa835e14a684 tempest-ServerActionsTestOtherB-626557989 tempest-ServerActionsTestOtherB-626557989-project-member] [instance: eeae9a99-e184-49ea-880f-0a429d893129] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.469188] nova-conductor[52800]: Traceback (most recent call last): [ 637.469188] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.469188] nova-conductor[52800]: return func(*args, **kwargs) [ 637.469188] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.469188] nova-conductor[52800]: selections = self._select_destinations( [ 637.469188] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.469188] nova-conductor[52800]: selections = self._schedule( [ 637.469188] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.469188] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 637.469188] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.469188] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 637.469188] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.469188] nova-conductor[52800]: ERROR nova.conductor.manager [ 637.480160] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.482108] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.482108] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.528771] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: 98b02f09-e666-421c-9a7f-baa6aa8ed7f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 637.530938] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.530938] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.530938] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.533491] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 637.533491] nova-conductor[52800]: Traceback (most recent call last): [ 637.533491] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.533491] nova-conductor[52800]: return func(*args, **kwargs) [ 637.533491] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.533491] nova-conductor[52800]: selections = self._select_destinations( [ 637.533491] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.533491] nova-conductor[52800]: selections = self._schedule( [ 637.533491] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.533491] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 637.533491] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.533491] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 637.533491] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.533491] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.534247] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-75b5674a-8bca-4684-a236-8e3edee29fae tempest-MigrationsAdminTest-1656569401 tempest-MigrationsAdminTest-1656569401-project-member] [instance: 98b02f09-e666-421c-9a7f-baa6aa8ed7f7] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.745962] nova-conductor[52799]: Traceback (most recent call last): [ 637.745962] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.745962] nova-conductor[52799]: return func(*args, **kwargs) [ 637.745962] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.745962] nova-conductor[52799]: selections = self._select_destinations( [ 637.745962] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.745962] nova-conductor[52799]: selections = self._schedule( [ 637.745962] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.745962] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 637.745962] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.745962] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 637.745962] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.745962] nova-conductor[52799]: ERROR nova.conductor.manager [ 637.755252] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.755471] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.755639] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.804511] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] [instance: b7b62f1a-ec58-4f3f-9551-2643486c9a06] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 637.805822] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.805822] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.806064] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.811349] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 637.811349] nova-conductor[52799]: Traceback (most recent call last): [ 637.811349] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.811349] nova-conductor[52799]: return func(*args, **kwargs) [ 637.811349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.811349] nova-conductor[52799]: selections = self._select_destinations( [ 637.811349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.811349] nova-conductor[52799]: selections = self._schedule( [ 637.811349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.811349] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 637.811349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.811349] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 637.811349] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.811349] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.811349] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] [instance: b7b62f1a-ec58-4f3f-9551-2643486c9a06] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.835925] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.836180] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.836374] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.878068] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] [instance: 9c20f3ba-9b92-40cf-892e-676947d47277] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 637.878811] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.879023] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.879191] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.882345] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 637.882345] nova-conductor[52799]: Traceback (most recent call last): [ 637.882345] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.882345] nova-conductor[52799]: return func(*args, **kwargs) [ 637.882345] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.882345] nova-conductor[52799]: selections = self._select_destinations( [ 637.882345] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.882345] nova-conductor[52799]: selections = self._schedule( [ 637.882345] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.882345] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 637.882345] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.882345] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 637.882345] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.882345] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.884058] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] [instance: 9c20f3ba-9b92-40cf-892e-676947d47277] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.912303] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.912695] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.912695] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.954154] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] [instance: 8a43915f-0fb8-4bca-ac72-30d4799c4598] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 637.954154] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 637.955834] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 637.955834] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 637.960171] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 637.960171] nova-conductor[52799]: Traceback (most recent call last): [ 637.960171] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 637.960171] nova-conductor[52799]: return func(*args, **kwargs) [ 637.960171] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 637.960171] nova-conductor[52799]: selections = self._select_destinations( [ 637.960171] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 637.960171] nova-conductor[52799]: selections = self._schedule( [ 637.960171] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 637.960171] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 637.960171] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 637.960171] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 637.960171] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 637.960171] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 637.960171] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-979f52d5-2704-410c-a030-533088e0674f tempest-ListServersNegativeTestJSON-445050069 tempest-ListServersNegativeTestJSON-445050069-project-member] [instance: 8a43915f-0fb8-4bca-ac72-30d4799c4598] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.017606] nova-conductor[52800]: Traceback (most recent call last): [ 638.017606] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 638.017606] nova-conductor[52800]: return func(*args, **kwargs) [ 638.017606] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 638.017606] nova-conductor[52800]: selections = self._select_destinations( [ 638.017606] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 638.017606] nova-conductor[52800]: selections = self._schedule( [ 638.017606] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 638.017606] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 638.017606] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 638.017606] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 638.017606] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.017606] nova-conductor[52800]: ERROR nova.conductor.manager [ 638.026824] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.027111] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.027292] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.073998] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] [instance: 2baf0578-0f53-4cc3-98dd-6e77cc05172f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 638.074711] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.074918] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.075104] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.078117] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 638.078117] nova-conductor[52800]: Traceback (most recent call last): [ 638.078117] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 638.078117] nova-conductor[52800]: return func(*args, **kwargs) [ 638.078117] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 638.078117] nova-conductor[52800]: selections = self._select_destinations( [ 638.078117] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 638.078117] nova-conductor[52800]: selections = self._schedule( [ 638.078117] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 638.078117] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 638.078117] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 638.078117] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 638.078117] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 638.078117] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.078633] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-afe55a09-b0be-43b7-abe3-c894d28bbd20 tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] [instance: 2baf0578-0f53-4cc3-98dd-6e77cc05172f] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.878351] nova-conductor[52799]: Traceback (most recent call last): [ 638.878351] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 638.878351] nova-conductor[52799]: return func(*args, **kwargs) [ 638.878351] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 638.878351] nova-conductor[52799]: selections = self._select_destinations( [ 638.878351] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 638.878351] nova-conductor[52799]: selections = self._schedule( [ 638.878351] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 638.878351] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 638.878351] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 638.878351] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 638.878351] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.878351] nova-conductor[52799]: ERROR nova.conductor.manager [ 638.888475] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.888852] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.889013] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.939725] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] [instance: 570f2695-73bf-433a-aac6-757a8c0cc2be] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 638.940520] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 638.940729] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 638.940896] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 638.944349] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 638.944349] nova-conductor[52799]: Traceback (most recent call last): [ 638.944349] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 638.944349] nova-conductor[52799]: return func(*args, **kwargs) [ 638.944349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 638.944349] nova-conductor[52799]: selections = self._select_destinations( [ 638.944349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 638.944349] nova-conductor[52799]: selections = self._schedule( [ 638.944349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 638.944349] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 638.944349] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 638.944349] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 638.944349] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 638.944349] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 638.944952] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-ad904beb-39e1-46f2-915a-d3f71408a48e tempest-VolumesAdminNegativeTest-1823021191 tempest-VolumesAdminNegativeTest-1823021191-project-member] [instance: 570f2695-73bf-433a-aac6-757a8c0cc2be] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 640.205646] nova-conductor[52800]: Traceback (most recent call last): [ 640.205646] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 640.205646] nova-conductor[52800]: return func(*args, **kwargs) [ 640.205646] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 640.205646] nova-conductor[52800]: selections = self._select_destinations( [ 640.205646] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 640.205646] nova-conductor[52800]: selections = self._schedule( [ 640.205646] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 640.205646] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 640.205646] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 640.205646] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 640.205646] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.205646] nova-conductor[52800]: ERROR nova.conductor.manager [ 640.212896] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 640.213135] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 640.213357] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 640.264283] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] [instance: 2f448ac7-9602-4f67-86d2-5a08f2e8073d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 640.264995] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 640.265222] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 640.265392] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 640.268273] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 640.268273] nova-conductor[52800]: Traceback (most recent call last): [ 640.268273] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 640.268273] nova-conductor[52800]: return func(*args, **kwargs) [ 640.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 640.268273] nova-conductor[52800]: selections = self._select_destinations( [ 640.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 640.268273] nova-conductor[52800]: selections = self._schedule( [ 640.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 640.268273] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 640.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 640.268273] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 640.268273] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 640.268273] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 640.268858] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-46552b67-e28c-4402-a7bc-8109cb4e6d4a tempest-AttachInterfacesUnderV243Test-691100930 tempest-AttachInterfacesUnderV243Test-691100930-project-member] [instance: 2f448ac7-9602-4f67-86d2-5a08f2e8073d] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 641.384824] nova-conductor[52799]: Traceback (most recent call last): [ 641.384824] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 641.384824] nova-conductor[52799]: return func(*args, **kwargs) [ 641.384824] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 641.384824] nova-conductor[52799]: selections = self._select_destinations( [ 641.384824] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 641.384824] nova-conductor[52799]: selections = self._schedule( [ 641.384824] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 641.384824] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 641.384824] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 641.384824] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 641.384824] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.384824] nova-conductor[52799]: ERROR nova.conductor.manager [ 641.397725] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 641.397725] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 641.397861] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 641.546181] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] [instance: 12f98059-926e-4975-bb9f-140a81140a98] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 641.547345] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 641.547345] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 641.547556] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 641.556165] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 641.556165] nova-conductor[52799]: Traceback (most recent call last): [ 641.556165] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 641.556165] nova-conductor[52799]: return func(*args, **kwargs) [ 641.556165] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 641.556165] nova-conductor[52799]: selections = self._select_destinations( [ 641.556165] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 641.556165] nova-conductor[52799]: selections = self._schedule( [ 641.556165] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 641.556165] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 641.556165] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 641.556165] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 641.556165] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 641.556165] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 641.556165] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-62ee7594-9004-41c3-be79-aae9b0a1282f tempest-ServerActionsTestJSON-1327811557 tempest-ServerActionsTestJSON-1327811557-project-member] [instance: 12f98059-926e-4975-bb9f-140a81140a98] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 641.980035] nova-conductor[52800]: Traceback (most recent call last): [ 641.980035] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 641.980035] nova-conductor[52800]: return func(*args, **kwargs) [ 641.980035] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 641.980035] nova-conductor[52800]: selections = self._select_destinations( [ 641.980035] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 641.980035] nova-conductor[52800]: selections = self._schedule( [ 641.980035] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 641.980035] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 641.980035] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 641.980035] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 641.980035] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.980035] nova-conductor[52800]: ERROR nova.conductor.manager [ 641.989042] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 641.989300] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 641.989472] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 642.044651] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] [instance: 6803e9ad-3c77-4ab0-8330-52fd4c143539] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 642.045249] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 642.045536] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 642.045735] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 642.049842] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 642.049842] nova-conductor[52800]: Traceback (most recent call last): [ 642.049842] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 642.049842] nova-conductor[52800]: return func(*args, **kwargs) [ 642.049842] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 642.049842] nova-conductor[52800]: selections = self._select_destinations( [ 642.049842] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 642.049842] nova-conductor[52800]: selections = self._schedule( [ 642.049842] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 642.049842] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 642.049842] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 642.049842] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 642.049842] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 642.049842] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 642.050616] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-58ea8be3-d439-467a-82d5-9770a1ca03ac tempest-ListImageFiltersTestJSON-70896247 tempest-ListImageFiltersTestJSON-70896247-project-member] [instance: 6803e9ad-3c77-4ab0-8330-52fd4c143539] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 644.640486] nova-conductor[52799]: Traceback (most recent call last): [ 644.640486] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 644.640486] nova-conductor[52799]: return func(*args, **kwargs) [ 644.640486] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 644.640486] nova-conductor[52799]: selections = self._select_destinations( [ 644.640486] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 644.640486] nova-conductor[52799]: selections = self._schedule( [ 644.640486] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 644.640486] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 644.640486] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 644.640486] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 644.640486] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.640486] nova-conductor[52799]: ERROR nova.conductor.manager [ 644.647410] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 644.647597] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 644.647765] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 644.693482] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 94c10ac0-bca0-4e76-8f47-f191809c7386] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 644.693933] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 644.694193] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 644.694411] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 644.699293] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 644.699293] nova-conductor[52799]: Traceback (most recent call last): [ 644.699293] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 644.699293] nova-conductor[52799]: return func(*args, **kwargs) [ 644.699293] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 644.699293] nova-conductor[52799]: selections = self._select_destinations( [ 644.699293] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 644.699293] nova-conductor[52799]: selections = self._schedule( [ 644.699293] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 644.699293] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 644.699293] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 644.699293] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 644.699293] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 644.699293] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 644.699860] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5d6cedec-fbbf-4fcc-93f0-1a8d332e2d5c tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: 94c10ac0-bca0-4e76-8f47-f191809c7386] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 647.730760] nova-conductor[52800]: Traceback (most recent call last): [ 647.730760] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 647.730760] nova-conductor[52800]: return func(*args, **kwargs) [ 647.730760] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 647.730760] nova-conductor[52800]: selections = self._select_destinations( [ 647.730760] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 647.730760] nova-conductor[52800]: selections = self._schedule( [ 647.730760] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 647.730760] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 647.730760] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 647.730760] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 647.730760] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.730760] nova-conductor[52800]: ERROR nova.conductor.manager [ 647.741532] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 647.741758] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 647.742423] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 647.805222] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] [instance: 8db450a6-1ca0-40d3-9350-a637df87d73c] block_device_mapping [BlockDeviceMapping(attachment_id=94f917ea-22f4-4e43-8a8d-aad773341019,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b616fd9d-c82f-4fa6-8cf6-6fe55c9dbf4f',volume_size=1,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 647.805982] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 647.806082] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 647.806239] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 647.814826] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 647.814826] nova-conductor[52800]: Traceback (most recent call last): [ 647.814826] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 647.814826] nova-conductor[52800]: return func(*args, **kwargs) [ 647.814826] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 647.814826] nova-conductor[52800]: selections = self._select_destinations( [ 647.814826] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 647.814826] nova-conductor[52800]: selections = self._schedule( [ 647.814826] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 647.814826] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 647.814826] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 647.814826] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 647.814826] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 647.814826] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 647.815851] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-f898dce4-2a4e-494b-bd4f-e364d154b03c tempest-ServersTestBootFromVolume-1072843563 tempest-ServersTestBootFromVolume-1072843563-project-member] [instance: 8db450a6-1ca0-40d3-9350-a637df87d73c] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.662125] nova-conductor[52799]: Traceback (most recent call last): [ 651.662125] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 651.662125] nova-conductor[52799]: return func(*args, **kwargs) [ 651.662125] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 651.662125] nova-conductor[52799]: selections = self._select_destinations( [ 651.662125] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 651.662125] nova-conductor[52799]: selections = self._schedule( [ 651.662125] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 651.662125] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 651.662125] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 651.662125] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 651.662125] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.662125] nova-conductor[52799]: ERROR nova.conductor.manager [ 651.675683] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 651.675683] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 651.675683] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 651.731062] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] [instance: 60ff0caf-22be-402f-a8a5-2773473bb595] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 651.734019] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 651.734019] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 651.734019] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 651.736245] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 651.736245] nova-conductor[52799]: Traceback (most recent call last): [ 651.736245] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 651.736245] nova-conductor[52799]: return func(*args, **kwargs) [ 651.736245] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 651.736245] nova-conductor[52799]: selections = self._select_destinations( [ 651.736245] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 651.736245] nova-conductor[52799]: selections = self._schedule( [ 651.736245] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 651.736245] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 651.736245] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 651.736245] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 651.736245] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 651.736245] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 651.737435] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-138d04e6-8688-4e12-bdd8-dd2fe2b10960 tempest-ImagesOneServerNegativeTestJSON-1401557552 tempest-ImagesOneServerNegativeTestJSON-1401557552-project-member] [instance: 60ff0caf-22be-402f-a8a5-2773473bb595] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 652.613013] nova-conductor[52800]: Traceback (most recent call last): [ 652.613013] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 652.613013] nova-conductor[52800]: return func(*args, **kwargs) [ 652.613013] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 652.613013] nova-conductor[52800]: selections = self._select_destinations( [ 652.613013] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 652.613013] nova-conductor[52800]: selections = self._schedule( [ 652.613013] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 652.613013] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 652.613013] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 652.613013] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 652.613013] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.613013] nova-conductor[52800]: ERROR nova.conductor.manager [ 652.622949] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 652.623557] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 652.623794] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 652.671858] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 2238169e-62f9-4d10-8d24-98a455e2be3d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 652.672623] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 652.672840] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 652.673027] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 652.676299] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 652.676299] nova-conductor[52800]: Traceback (most recent call last): [ 652.676299] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 652.676299] nova-conductor[52800]: return func(*args, **kwargs) [ 652.676299] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 652.676299] nova-conductor[52800]: selections = self._select_destinations( [ 652.676299] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 652.676299] nova-conductor[52800]: selections = self._schedule( [ 652.676299] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 652.676299] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 652.676299] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 652.676299] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 652.676299] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 652.676299] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 652.677108] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-f0c3c2e3-70b9-4508-bd71-c75fe6064932 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 2238169e-62f9-4d10-8d24-98a455e2be3d] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 653.472548] nova-conductor[52799]: Traceback (most recent call last): [ 653.472548] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 653.472548] nova-conductor[52799]: return func(*args, **kwargs) [ 653.472548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 653.472548] nova-conductor[52799]: selections = self._select_destinations( [ 653.472548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 653.472548] nova-conductor[52799]: selections = self._schedule( [ 653.472548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 653.472548] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 653.472548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 653.472548] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 653.472548] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.472548] nova-conductor[52799]: ERROR nova.conductor.manager [ 653.479552] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 653.479779] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 653.479951] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 653.533402] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] [instance: c83f2531-e9ca-439d-b46f-33b3c7603a69] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 653.534331] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 653.534570] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 653.534742] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 653.542089] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 653.542089] nova-conductor[52799]: Traceback (most recent call last): [ 653.542089] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 653.542089] nova-conductor[52799]: return func(*args, **kwargs) [ 653.542089] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 653.542089] nova-conductor[52799]: selections = self._select_destinations( [ 653.542089] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 653.542089] nova-conductor[52799]: selections = self._schedule( [ 653.542089] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 653.542089] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 653.542089] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 653.542089] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 653.542089] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 653.542089] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 653.542742] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d631864b-0dd4-4cdc-b74e-7396beed2cc6 tempest-InstanceActionsV221TestJSON-1778790919 tempest-InstanceActionsV221TestJSON-1778790919-project-member] [instance: c83f2531-e9ca-439d-b46f-33b3c7603a69] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 658.461956] nova-conductor[52800]: Traceback (most recent call last): [ 658.461956] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 658.461956] nova-conductor[52800]: return func(*args, **kwargs) [ 658.461956] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 658.461956] nova-conductor[52800]: selections = self._select_destinations( [ 658.461956] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 658.461956] nova-conductor[52800]: selections = self._schedule( [ 658.461956] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 658.461956] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 658.461956] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 658.461956] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 658.461956] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.461956] nova-conductor[52800]: ERROR nova.conductor.manager [ 658.472932] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 658.473167] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 658.473347] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 658.524133] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] [instance: 39537c41-7d28-443c-85f2-b5e170b6b0d2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 658.524676] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 658.525761] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 658.525761] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 658.528319] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 658.528319] nova-conductor[52800]: Traceback (most recent call last): [ 658.528319] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 658.528319] nova-conductor[52800]: return func(*args, **kwargs) [ 658.528319] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 658.528319] nova-conductor[52800]: selections = self._select_destinations( [ 658.528319] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 658.528319] nova-conductor[52800]: selections = self._schedule( [ 658.528319] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 658.528319] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 658.528319] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 658.528319] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 658.528319] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 658.528319] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 658.528866] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-b942a7e4-b34b-4577-813d-9da94d7294c4 tempest-ServersNegativeTestMultiTenantJSON-947424472 tempest-ServersNegativeTestMultiTenantJSON-947424472-project-member] [instance: 39537c41-7d28-443c-85f2-b5e170b6b0d2] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.110261] nova-conductor[52799]: Traceback (most recent call last): [ 660.110261] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 660.110261] nova-conductor[52799]: return func(*args, **kwargs) [ 660.110261] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 660.110261] nova-conductor[52799]: selections = self._select_destinations( [ 660.110261] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 660.110261] nova-conductor[52799]: selections = self._schedule( [ 660.110261] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 660.110261] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 660.110261] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 660.110261] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 660.110261] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.110261] nova-conductor[52799]: ERROR nova.conductor.manager [ 660.119971] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 660.122175] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 660.122175] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 660.180935] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] [instance: c34cfa49-bc27-47e4-8b66-fb9b3a60537e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 660.181745] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 660.181956] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 660.182136] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 660.189472] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 660.189472] nova-conductor[52799]: Traceback (most recent call last): [ 660.189472] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 660.189472] nova-conductor[52799]: return func(*args, **kwargs) [ 660.189472] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 660.189472] nova-conductor[52799]: selections = self._select_destinations( [ 660.189472] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 660.189472] nova-conductor[52799]: selections = self._schedule( [ 660.189472] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 660.189472] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 660.189472] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 660.189472] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 660.189472] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 660.189472] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 660.190877] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-10ad097b-1b0f-4e9d-8c1f-71846c93624f tempest-InstanceActionsTestJSON-177862478 tempest-InstanceActionsTestJSON-177862478-project-member] [instance: c34cfa49-bc27-47e4-8b66-fb9b3a60537e] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 662.069021] nova-conductor[52800]: Traceback (most recent call last): [ 662.069021] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 662.069021] nova-conductor[52800]: return func(*args, **kwargs) [ 662.069021] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 662.069021] nova-conductor[52800]: selections = self._select_destinations( [ 662.069021] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 662.069021] nova-conductor[52800]: selections = self._schedule( [ 662.069021] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 662.069021] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 662.069021] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 662.069021] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 662.069021] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.069021] nova-conductor[52800]: ERROR nova.conductor.manager [ 662.076297] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 662.076912] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 662.077395] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 662.128822] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: c019a6d1-1d6b-4af0-9124-36daa10e5813] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 662.129791] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 662.131463] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 662.131463] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 662.134496] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 662.134496] nova-conductor[52800]: Traceback (most recent call last): [ 662.134496] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 662.134496] nova-conductor[52800]: return func(*args, **kwargs) [ 662.134496] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 662.134496] nova-conductor[52800]: selections = self._select_destinations( [ 662.134496] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 662.134496] nova-conductor[52800]: selections = self._schedule( [ 662.134496] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 662.134496] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 662.134496] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 662.134496] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 662.134496] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 662.134496] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 662.135054] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-3e81727d-d345-4ad9-8326-3329600a6291 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: c019a6d1-1d6b-4af0-9124-36daa10e5813] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 666.481880] nova-conductor[52799]: Traceback (most recent call last): [ 666.481880] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 666.481880] nova-conductor[52799]: return func(*args, **kwargs) [ 666.481880] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 666.481880] nova-conductor[52799]: selections = self._select_destinations( [ 666.481880] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 666.481880] nova-conductor[52799]: selections = self._schedule( [ 666.481880] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 666.481880] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 666.481880] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 666.481880] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 666.481880] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.481880] nova-conductor[52799]: ERROR nova.conductor.manager [ 666.493028] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 666.493108] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 666.493296] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 666.555262] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: d725ec92-b70d-4567-8f9a-ac90a71066fc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 666.557649] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 666.557649] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 666.557649] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 666.561175] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 666.561175] nova-conductor[52799]: Traceback (most recent call last): [ 666.561175] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 666.561175] nova-conductor[52799]: return func(*args, **kwargs) [ 666.561175] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 666.561175] nova-conductor[52799]: selections = self._select_destinations( [ 666.561175] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 666.561175] nova-conductor[52799]: selections = self._schedule( [ 666.561175] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 666.561175] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 666.561175] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 666.561175] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 666.561175] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 666.561175] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 666.561758] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-4483b7a6-787d-4174-935d-32d072eafbbd tempest-ImagesTestJSON-1759893354 tempest-ImagesTestJSON-1759893354-project-member] [instance: d725ec92-b70d-4567-8f9a-ac90a71066fc] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 669.273634] nova-conductor[52800]: Traceback (most recent call last): [ 669.273634] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 669.273634] nova-conductor[52800]: return func(*args, **kwargs) [ 669.273634] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 669.273634] nova-conductor[52800]: selections = self._select_destinations( [ 669.273634] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 669.273634] nova-conductor[52800]: selections = self._schedule( [ 669.273634] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 669.273634] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 669.273634] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 669.273634] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 669.273634] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: ERROR nova.conductor.manager [ 669.273634] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 669.273634] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 669.286987] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 669.337810] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: 420fda1a-e63f-40d0-9017-02e47576a94a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 669.337810] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 669.337810] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 669.337810] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 669.357667] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 669.357667] nova-conductor[52800]: Traceback (most recent call last): [ 669.357667] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 669.357667] nova-conductor[52800]: return func(*args, **kwargs) [ 669.357667] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 669.357667] nova-conductor[52800]: selections = self._select_destinations( [ 669.357667] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 669.357667] nova-conductor[52800]: selections = self._schedule( [ 669.357667] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 669.357667] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 669.357667] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 669.357667] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 669.357667] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 669.357667] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 669.357667] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-9b1525f7-b524-4862-add9-8cf08c472273 tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: 420fda1a-e63f-40d0-9017-02e47576a94a] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 671.437149] nova-conductor[52799]: Traceback (most recent call last): [ 671.437149] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 671.437149] nova-conductor[52799]: return func(*args, **kwargs) [ 671.437149] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 671.437149] nova-conductor[52799]: selections = self._select_destinations( [ 671.437149] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 671.437149] nova-conductor[52799]: selections = self._schedule( [ 671.437149] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 671.437149] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 671.437149] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 671.437149] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 671.437149] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.437149] nova-conductor[52799]: ERROR nova.conductor.manager [ 671.448863] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 671.449218] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 671.449479] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 671.512392] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] [instance: 9f8485b1-6863-4f17-aff8-e5e878ac521b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 671.513573] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 671.513902] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 671.514350] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 671.518093] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 671.518093] nova-conductor[52799]: Traceback (most recent call last): [ 671.518093] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 671.518093] nova-conductor[52799]: return func(*args, **kwargs) [ 671.518093] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 671.518093] nova-conductor[52799]: selections = self._select_destinations( [ 671.518093] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 671.518093] nova-conductor[52799]: selections = self._schedule( [ 671.518093] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 671.518093] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 671.518093] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 671.518093] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 671.518093] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 671.518093] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 671.518790] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-0802554e-ee14-4b64-94b7-db29bd57753f tempest-ServersTestJSON-2042308697 tempest-ServersTestJSON-2042308697-project-member] [instance: 9f8485b1-6863-4f17-aff8-e5e878ac521b] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 674.223985] nova-conductor[52800]: Traceback (most recent call last): [ 674.223985] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 674.223985] nova-conductor[52800]: return func(*args, **kwargs) [ 674.223985] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 674.223985] nova-conductor[52800]: selections = self._select_destinations( [ 674.223985] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 674.223985] nova-conductor[52800]: selections = self._schedule( [ 674.223985] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 674.223985] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 674.223985] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 674.223985] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 674.223985] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.223985] nova-conductor[52800]: ERROR nova.conductor.manager [ 674.235437] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 674.236172] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 674.236172] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 674.293727] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 474624ad-0060-410f-8a65-50f3fa3cc4d0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 674.296670] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 674.296670] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 674.296670] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 674.302123] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 674.302123] nova-conductor[52800]: Traceback (most recent call last): [ 674.302123] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 674.302123] nova-conductor[52800]: return func(*args, **kwargs) [ 674.302123] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 674.302123] nova-conductor[52800]: selections = self._select_destinations( [ 674.302123] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 674.302123] nova-conductor[52800]: selections = self._schedule( [ 674.302123] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 674.302123] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 674.302123] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 674.302123] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 674.302123] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 674.302123] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 674.303174] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-a9f8336c-7661-4643-965b-fc0a899fdd61 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 474624ad-0060-410f-8a65-50f3fa3cc4d0] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 677.525748] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 677.540850] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.540850] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.541017] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.578516] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.579560] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.579560] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.583019] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.583019] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.583019] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.590727] nova-conductor[52800]: DEBUG nova.quota [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Getting quotas for project 8d75debce2fd4b2492cc02aeb2fed7fc. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 677.595027] nova-conductor[52800]: DEBUG nova.quota [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Getting quotas for user f7221f06a45d45f2a34ab3bdd869113d and project 8d75debce2fd4b2492cc02aeb2fed7fc. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 677.600362] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: e46ad9d2-d215-4205-b0c1-44726b08cb45] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 677.601061] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.601528] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.601854] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.605023] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: e46ad9d2-d215-4205-b0c1-44726b08cb45] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 677.606033] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.606033] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.606949] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.620027] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.620256] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.620426] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.872620] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Took 0.19 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 677.892102] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.892220] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.892367] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.933113] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.933351] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.933525] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.934280] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.934484] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.934650] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.950846] nova-conductor[52799]: DEBUG nova.quota [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Getting quotas for project af6d8fe22e704a3aa476242b6f6ef896. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 677.956021] nova-conductor[52799]: DEBUG nova.quota [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Getting quotas for user 4789db843a4c4dd298c1b998a782261a and project af6d8fe22e704a3aa476242b6f6ef896. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 677.963993] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: 7466dfd3-8756-40eb-91fd-c87f16b627ef] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 677.965644] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.965954] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.966234] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.970793] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: 7466dfd3-8756-40eb-91fd-c87f16b627ef] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 677.970979] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.971395] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.971395] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 677.994614] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 677.994981] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 677.995178] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.284024] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 680.295432] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.296590] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.296590] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.343219] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.343219] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.343219] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.343219] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.343219] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.343219] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.353676] nova-conductor[52799]: DEBUG nova.quota [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Getting quotas for project 39d93795676543a287744d7fde7dde5c. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 680.363955] nova-conductor[52799]: DEBUG nova.quota [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Getting quotas for user b0eda48516c5499db963c96ad8a8a0f2 and project 39d93795676543a287744d7fde7dde5c. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 680.369733] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] [instance: 68545276-63f2-4baf-8110-d3cc71686682] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 680.370320] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.370584] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.370788] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.373915] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] [instance: 68545276-63f2-4baf-8110-d3cc71686682] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 680.375195] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.375446] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.375731] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.394309] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 680.395351] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.395351] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.395351] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.408575] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.408806] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.409017] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.476223] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.476223] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.476223] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.476223] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.476223] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.476223] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.496125] nova-conductor[52800]: DEBUG nova.quota [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Getting quotas for project af6d8fe22e704a3aa476242b6f6ef896. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 680.503023] nova-conductor[52800]: DEBUG nova.quota [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Getting quotas for user 4789db843a4c4dd298c1b998a782261a and project af6d8fe22e704a3aa476242b6f6ef896. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 680.510427] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: b1361aa5-9bbd-4891-b74f-a0afd90b0bd6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 680.511520] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.511653] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.511784] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.515672] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: b1361aa5-9bbd-4891-b74f-a0afd90b0bd6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 680.516608] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.516893] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.517144] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 680.540586] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 680.540586] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 680.540586] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.124921] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 683.144584] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.145536] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.145743] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.190048] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.190048] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.190184] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.190470] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.190657] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.190810] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.206378] nova-conductor[52799]: DEBUG nova.quota [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Getting quotas for project 0be6718d0cbe4351a06b59576311c7f8. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 683.208898] nova-conductor[52799]: DEBUG nova.quota [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Getting quotas for user 56bb638542d440639e1a38b10e80fb1e and project 0be6718d0cbe4351a06b59576311c7f8. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 683.219145] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 4c545ed0-7442-43db-a96f-4d7f1b785c4d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 683.219145] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.219145] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.219145] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.223092] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 4c545ed0-7442-43db-a96f-4d7f1b785c4d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 683.223839] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.224055] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.224221] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 683.244891] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 683.244891] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 683.245061] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.116811] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 685.131516] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.132066] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.132066] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.162555] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.162777] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.162948] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.163376] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.163499] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.163659] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.175677] nova-conductor[52800]: DEBUG nova.quota [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Getting quotas for project a0e821828b384a40a78522e538c9dcdf. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 685.178874] nova-conductor[52800]: DEBUG nova.quota [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Getting quotas for user 6dbce89f566b4a8eaaa8977f3c593ceb and project a0e821828b384a40a78522e538c9dcdf. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 685.186598] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] [instance: 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 685.187300] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.187539] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.187843] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.190783] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] [instance: 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 685.191437] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.191653] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.193648] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 685.212953] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 685.213179] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 685.213344] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 690.005059] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Took 0.13 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 690.017262] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 690.017262] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 690.017377] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 690.046740] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 690.046740] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 690.046740] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 690.046893] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 690.046984] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 690.047250] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 690.056263] nova-conductor[52799]: DEBUG nova.quota [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Getting quotas for project 39a67a0257094a479656d0975c7f2127. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 690.058790] nova-conductor[52799]: DEBUG nova.quota [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Getting quotas for user 5c9431bd76694bbab69bffb93531532f and project 39a67a0257094a479656d0975c7f2127. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 690.065638] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] [instance: 5980cfa0-bdd6-4fca-a605-c857e0e7b886] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 690.066140] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 690.066345] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 690.066520] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 690.069685] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] [instance: 5980cfa0-bdd6-4fca-a605-c857e0e7b886] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 690.070358] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 690.070559] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 690.070726] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 690.087073] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 690.087252] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 690.087405] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 691.296563] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 691.310209] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 691.310209] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 691.310789] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 691.356642] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 691.356910] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 691.357132] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 691.357952] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 691.358235] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 691.358555] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 691.369030] nova-conductor[52800]: DEBUG nova.quota [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Getting quotas for project 1d1843cb3adc4fe59efbc5dd6a7c0f32. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 691.371928] nova-conductor[52800]: DEBUG nova.quota [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Getting quotas for user f14d545d067e4b77b659505dae36811d and project 1d1843cb3adc4fe59efbc5dd6a7c0f32. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 691.378476] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] [instance: 5ada08c2-ea12-4b16-9384-af545c8e06aa] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 691.379155] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 691.379442] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 691.379684] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 691.384400] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] [instance: 5ada08c2-ea12-4b16-9384-af545c8e06aa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 691.385111] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 691.385322] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 691.385492] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 691.402851] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 691.403091] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 691.403341] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 693.382041] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 693.401267] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 693.402084] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 693.402393] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 693.446793] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 693.447050] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 693.448321] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 693.448321] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 693.448321] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 693.448321] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 693.461881] nova-conductor[52799]: DEBUG nova.quota [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Getting quotas for project df737b66de7f4333a492593a77abd42d. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 693.467615] nova-conductor[52799]: DEBUG nova.quota [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Getting quotas for user cb884ffadf2145ad959c0b159464b7b9 and project df737b66de7f4333a492593a77abd42d. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 693.477431] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] [instance: 7e356a2e-b299-4801-af74-f536a12489fc] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 693.478070] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 693.478184] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 693.478353] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 693.482615] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] [instance: 7e356a2e-b299-4801-af74-f536a12489fc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 693.483376] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 693.483586] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 693.483777] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 693.502570] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 693.503038] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 693.503215] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 694.394108] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: e46ad9d2-d215-4205-b0c1-44726b08cb45] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 916b0b39-e552-4059-bbaf-20e8e06f1998, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e46ad9d2-d215-4205-b0c1-44726b08cb45 was re-scheduled: Binding failed for port 916b0b39-e552-4059-bbaf-20e8e06f1998, please check neutron logs for more information.\n'] [ 694.397019] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 694.397019] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e46ad9d2-d215-4205-b0c1-44726b08cb45.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e46ad9d2-d215-4205-b0c1-44726b08cb45. [ 694.397019] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: e46ad9d2-d215-4205-b0c1-44726b08cb45] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e46ad9d2-d215-4205-b0c1-44726b08cb45. [ 694.421537] nova-conductor[52799]: DEBUG nova.network.neutron [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: e46ad9d2-d215-4205-b0c1-44726b08cb45] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 694.503492] nova-conductor[52799]: DEBUG nova.network.neutron [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: e46ad9d2-d215-4205-b0c1-44726b08cb45] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 694.514557] nova-conductor[52799]: DEBUG nova.network.neutron [None req-f150b914-55df-4af9-b649-b789a293f473 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: e46ad9d2-d215-4205-b0c1-44726b08cb45] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 696.348561] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 696.361659] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.362142] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.362142] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.452869] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.453158] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.454030] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.454186] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.454379] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.454559] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.467644] nova-conductor[52800]: DEBUG nova.quota [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Getting quotas for project 15c6dd50db8b44d190cdaaf8e69222da. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 696.473871] nova-conductor[52800]: DEBUG nova.quota [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Getting quotas for user 38afd0b6a9a24556bd374e62b3363f3d and project 15c6dd50db8b44d190cdaaf8e69222da. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 696.482524] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: e52a3adf-4654-43cd-8613-749277053ea8] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 696.483043] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.483250] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.483443] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.486749] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: e52a3adf-4654-43cd-8613-749277053ea8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 696.489029] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.489029] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.489029] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.524018] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.524018] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.524018] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.662498] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 696.678984] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.679321] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.679408] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.722682] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.722907] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.723086] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.723435] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.723613] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.723766] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.733101] nova-conductor[52799]: DEBUG nova.quota [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Getting quotas for project 61a1fea64b51425d840cb421206a4191. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 696.735422] nova-conductor[52799]: DEBUG nova.quota [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Getting quotas for user ea6be32321a24a638ad7079b323457de and project 61a1fea64b51425d840cb421206a4191. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 696.742113] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] [instance: ef7b219e-437d-4b15-b559-ca5e2405efb2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 696.742536] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.742731] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.742894] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.749019] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] [instance: ef7b219e-437d-4b15-b559-ca5e2405efb2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 696.749019] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.749019] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.749019] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 696.760539] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 696.760767] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 696.760933] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.578893] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 4c545ed0-7442-43db-a96f-4d7f1b785c4d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6ea2b6b0-f8d9-4d8e-8406-96a2d5c43db7, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 4c545ed0-7442-43db-a96f-4d7f1b785c4d was re-scheduled: Binding failed for port 6ea2b6b0-f8d9-4d8e-8406-96a2d5c43db7, please check neutron logs for more information.\n'] [ 698.579723] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 698.579786] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4c545ed0-7442-43db-a96f-4d7f1b785c4d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4c545ed0-7442-43db-a96f-4d7f1b785c4d. [ 698.580081] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 4c545ed0-7442-43db-a96f-4d7f1b785c4d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 4c545ed0-7442-43db-a96f-4d7f1b785c4d. [ 698.603032] nova-conductor[52799]: DEBUG nova.network.neutron [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 4c545ed0-7442-43db-a96f-4d7f1b785c4d] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 698.657056] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 698.669860] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.670573] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.670573] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.698845] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.699078] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.699252] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.699632] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.699811] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.699999] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.708516] nova-conductor[52799]: DEBUG nova.quota [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Getting quotas for project f32917678ead4147ab017ac9de07a145. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 698.710835] nova-conductor[52799]: DEBUG nova.quota [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Getting quotas for user 2af3b82f197e4e7fbb0600b1eb0b34ef and project f32917678ead4147ab017ac9de07a145. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 698.716176] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 9bee98ef-48b4-47e6-8afb-e535e58e50cb] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 698.716606] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.716873] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.717047] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.719815] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 9bee98ef-48b4-47e6-8afb-e535e58e50cb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 698.720475] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.720700] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.720864] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.736211] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 698.736376] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 698.736485] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 698.833744] nova-conductor[52799]: DEBUG nova.network.neutron [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 4c545ed0-7442-43db-a96f-4d7f1b785c4d] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 698.838672] nova-conductor[52799]: DEBUG nova.network.neutron [None req-932f1f02-6b5d-44c3-a7cc-23843d41d098 tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 4c545ed0-7442-43db-a96f-4d7f1b785c4d] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 699.250034] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 699.274807] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 699.275051] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 699.275260] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 699.310301] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 699.310301] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 699.310301] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 699.310592] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 699.310628] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 699.310791] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 699.321246] nova-conductor[52799]: DEBUG nova.quota [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Getting quotas for project b1fece12693747c2b6e56f92f305881f. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 699.324158] nova-conductor[52799]: DEBUG nova.quota [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Getting quotas for user 4bf93c97450a44e5bf91e32b218736d2 and project b1fece12693747c2b6e56f92f305881f. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 699.331829] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] [instance: 9e88b24c-500d-4efb-8563-093dd4d0378d] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 699.331829] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 699.331829] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 699.331829] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 699.343610] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] [instance: 9e88b24c-500d-4efb-8563-093dd4d0378d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 699.344280] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 699.344481] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 699.344690] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 699.378167] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 699.378167] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 699.378167] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 702.650179] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] [instance: 5980cfa0-bdd6-4fca-a605-c857e0e7b886] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port c4b075f2-0c8a-430a-9b5a-9817c1efc2b3, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5980cfa0-bdd6-4fca-a605-c857e0e7b886 was re-scheduled: Binding failed for port c4b075f2-0c8a-430a-9b5a-9817c1efc2b3, please check neutron logs for more information.\n'] [ 702.650179] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 702.650179] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5980cfa0-bdd6-4fca-a605-c857e0e7b886.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5980cfa0-bdd6-4fca-a605-c857e0e7b886. [ 702.650179] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] [instance: 5980cfa0-bdd6-4fca-a605-c857e0e7b886] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5980cfa0-bdd6-4fca-a605-c857e0e7b886. [ 702.680907] nova-conductor[52800]: DEBUG nova.network.neutron [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] [instance: 5980cfa0-bdd6-4fca-a605-c857e0e7b886] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 702.734247] nova-conductor[52800]: DEBUG nova.network.neutron [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] [instance: 5980cfa0-bdd6-4fca-a605-c857e0e7b886] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 702.737076] nova-conductor[52800]: DEBUG nova.network.neutron [None req-6a659fcc-5f6c-43fa-af74-873b3ff2b4a5 tempest-ServerPasswordTestJSON-1791643214 tempest-ServerPasswordTestJSON-1791643214-project-member] [instance: 5980cfa0-bdd6-4fca-a605-c857e0e7b886] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 703.173125] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Took 0.27 seconds to select destinations for 2 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 703.192128] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.192593] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.192903] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.244415] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.244640] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.244823] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.270346] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.271073] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.271427] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.271851] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.272128] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.272397] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.289390] nova-conductor[52800]: DEBUG nova.quota [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Getting quotas for project a98b1fd8031545e381db0682e508fc18. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 703.294327] nova-conductor[52800]: DEBUG nova.quota [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Getting quotas for user 4cc2b0ed84534852a16f9fdd4a8977f7 and project a98b1fd8031545e381db0682e508fc18. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 703.302920] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: c7cb800a-3634-44e4-bb18-fab9d2e86c7e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 703.303495] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.303766] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.304008] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.306668] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: c7cb800a-3634-44e4-bb18-fab9d2e86c7e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 703.307366] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.307646] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.307889] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.323449] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.323756] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.323942] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.332979] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 8fcf260d-2796-4972-b217-95954e309a6e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 703.332979] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.332979] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.332979] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.336943] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 8fcf260d-2796-4972-b217-95954e309a6e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 703.336943] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.336943] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.336943] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.360302] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 703.360302] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 703.360402] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 703.768023] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] [instance: 5ada08c2-ea12-4b16-9384-af545c8e06aa] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 93c5b658-4e25-4747-b1bb-79ac51446057, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 5ada08c2-ea12-4b16-9384-af545c8e06aa was re-scheduled: Binding failed for port 93c5b658-4e25-4747-b1bb-79ac51446057, please check neutron logs for more information.\n'] [ 703.768580] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 703.768816] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5ada08c2-ea12-4b16-9384-af545c8e06aa.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5ada08c2-ea12-4b16-9384-af545c8e06aa. [ 703.769691] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] [instance: 5ada08c2-ea12-4b16-9384-af545c8e06aa] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 5ada08c2-ea12-4b16-9384-af545c8e06aa. [ 703.807412] nova-conductor[52800]: DEBUG nova.network.neutron [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] [instance: 5ada08c2-ea12-4b16-9384-af545c8e06aa] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 703.875654] nova-conductor[52800]: DEBUG nova.network.neutron [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] [instance: 5ada08c2-ea12-4b16-9384-af545c8e06aa] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 703.878279] nova-conductor[52800]: DEBUG nova.network.neutron [None req-73bbd370-d330-49ce-9037-38c4e2ad9cc4 tempest-ServerMetadataTestJSON-855428705 tempest-ServerMetadataTestJSON-855428705-project-member] [instance: 5ada08c2-ea12-4b16-9384-af545c8e06aa] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 704.117408] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Took 1.18 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 704.141642] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 704.141976] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 704.142199] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 704.218552] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 704.218717] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 704.218933] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 704.219334] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 704.219733] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 704.219733] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 704.227850] nova-conductor[52799]: DEBUG nova.quota [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Getting quotas for project 9af4fbea46444d81b8ed5dd844ce87d4. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 704.230330] nova-conductor[52799]: DEBUG nova.quota [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Getting quotas for user 37788e4056b84ab0b461767fad9e3955 and project 9af4fbea46444d81b8ed5dd844ce87d4. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 704.256218] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c9c2d371-978e-4037-ba78-9b44f40765bd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 704.256218] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 704.256218] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 704.256218] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 704.262289] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c9c2d371-978e-4037-ba78-9b44f40765bd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 704.263034] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 704.263179] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 704.263345] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 704.285365] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 704.285571] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 704.285731] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 705.324855] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Took 0.20 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 705.352127] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 705.352866] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 705.353208] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 705.401263] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 705.401552] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 705.401951] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 705.402256] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 705.402946] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 705.402946] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 705.420150] nova-conductor[52799]: DEBUG nova.quota [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Getting quotas for project 15c6dd50db8b44d190cdaaf8e69222da. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 705.423047] nova-conductor[52799]: DEBUG nova.quota [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Getting quotas for user 38afd0b6a9a24556bd374e62b3363f3d and project 15c6dd50db8b44d190cdaaf8e69222da. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 705.451968] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: 156cf534-81ca-4cc6-9b0d-2d245016c53c] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 705.452092] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 705.455590] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 705.455590] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 705.459746] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: 156cf534-81ca-4cc6-9b0d-2d245016c53c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 705.460430] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 705.460629] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 705.460792] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 705.495386] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 705.495386] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 705.495386] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 706.738032] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] [instance: 7e356a2e-b299-4801-af74-f536a12489fc] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port e641e291-cbd8-404d-ab1d-4dee8d7969cd, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 7e356a2e-b299-4801-af74-f536a12489fc was re-scheduled: Binding failed for port e641e291-cbd8-404d-ab1d-4dee8d7969cd, please check neutron logs for more information.\n'] [ 706.740664] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 706.740664] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7e356a2e-b299-4801-af74-f536a12489fc.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7e356a2e-b299-4801-af74-f536a12489fc. [ 706.740664] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] [instance: 7e356a2e-b299-4801-af74-f536a12489fc] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7e356a2e-b299-4801-af74-f536a12489fc. [ 706.777102] nova-conductor[52800]: DEBUG nova.network.neutron [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] [instance: 7e356a2e-b299-4801-af74-f536a12489fc] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 706.851610] nova-conductor[52800]: DEBUG nova.network.neutron [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] [instance: 7e356a2e-b299-4801-af74-f536a12489fc] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 706.856720] nova-conductor[52800]: DEBUG nova.network.neutron [None req-24b40525-bc39-4c67-8fb1-5b1623397163 tempest-ServersV294TestFqdnHostnames-1586253388 tempest-ServersV294TestFqdnHostnames-1586253388-project-member] [instance: 7e356a2e-b299-4801-af74-f536a12489fc] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 707.047412] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 707.079103] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 707.079340] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 707.079533] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 707.121274] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 707.121274] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 707.121274] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 707.121585] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 707.121767] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 707.121947] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 707.134482] nova-conductor[52800]: DEBUG nova.quota [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Getting quotas for project 9af4fbea46444d81b8ed5dd844ce87d4. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 707.138764] nova-conductor[52800]: DEBUG nova.quota [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Getting quotas for user 37788e4056b84ab0b461767fad9e3955 and project 9af4fbea46444d81b8ed5dd844ce87d4. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 707.154020] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 707.154020] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 707.154020] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 707.154020] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 707.158147] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 707.159053] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 707.159053] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 707.159193] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 707.182221] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 707.182452] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 707.182622] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 707.858568] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: e52a3adf-4654-43cd-8613-749277053ea8] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 53857d03-68f5-47d9-b7d9-b532a6d42fcf, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e52a3adf-4654-43cd-8613-749277053ea8 was re-scheduled: Binding failed for port 53857d03-68f5-47d9-b7d9-b532a6d42fcf, please check neutron logs for more information.\n'] [ 707.860065] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 707.860065] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e52a3adf-4654-43cd-8613-749277053ea8.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e52a3adf-4654-43cd-8613-749277053ea8. [ 707.861522] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: e52a3adf-4654-43cd-8613-749277053ea8] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e52a3adf-4654-43cd-8613-749277053ea8. [ 707.897138] nova-conductor[52800]: DEBUG nova.network.neutron [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: e52a3adf-4654-43cd-8613-749277053ea8] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 707.987030] nova-conductor[52800]: DEBUG nova.network.neutron [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: e52a3adf-4654-43cd-8613-749277053ea8] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 707.990785] nova-conductor[52800]: DEBUG nova.network.neutron [None req-e294a58d-5428-4f1a-816a-e9e5a9a27f37 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: e52a3adf-4654-43cd-8613-749277053ea8] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 708.712891] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] [instance: ef7b219e-437d-4b15-b559-ca5e2405efb2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 05e585fb-54d8-4a6a-b92f-a1b020c55e21, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ef7b219e-437d-4b15-b559-ca5e2405efb2 was re-scheduled: Binding failed for port 05e585fb-54d8-4a6a-b92f-a1b020c55e21, please check neutron logs for more information.\n'] [ 708.714058] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 708.715330] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ef7b219e-437d-4b15-b559-ca5e2405efb2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ef7b219e-437d-4b15-b559-ca5e2405efb2. [ 708.715330] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] [instance: ef7b219e-437d-4b15-b559-ca5e2405efb2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ef7b219e-437d-4b15-b559-ca5e2405efb2. [ 708.746565] nova-conductor[52799]: DEBUG nova.network.neutron [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] [instance: ef7b219e-437d-4b15-b559-ca5e2405efb2] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 708.788672] nova-conductor[52799]: DEBUG nova.network.neutron [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] [instance: ef7b219e-437d-4b15-b559-ca5e2405efb2] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 708.796949] nova-conductor[52799]: DEBUG nova.network.neutron [None req-85d39ece-81cb-4714-a505-602e6aa6d34a tempest-ServerRescueTestJSON-406518722 tempest-ServerRescueTestJSON-406518722-project-member] [instance: ef7b219e-437d-4b15-b559-ca5e2405efb2] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 710.065710] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 710.081556] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.081797] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.082492] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.123038] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.123274] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.123443] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.123786] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.123963] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.124140] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.134770] nova-conductor[52799]: DEBUG nova.quota [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Getting quotas for project 9af4fbea46444d81b8ed5dd844ce87d4. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 710.139145] nova-conductor[52799]: DEBUG nova.quota [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Getting quotas for user 37788e4056b84ab0b461767fad9e3955 and project 9af4fbea46444d81b8ed5dd844ce87d4. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 710.146597] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 710.147135] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.147426] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.147572] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.150719] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 710.151384] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.151569] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.151739] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.168017] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.168017] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.168017] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.355951] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 710.374334] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.374566] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.374734] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.413514] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.413806] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.413925] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.414359] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.414456] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.414591] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.426897] nova-conductor[52800]: DEBUG nova.quota [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Getting quotas for project 8d75debce2fd4b2492cc02aeb2fed7fc. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 710.432567] nova-conductor[52800]: DEBUG nova.quota [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Getting quotas for user f7221f06a45d45f2a34ab3bdd869113d and project 8d75debce2fd4b2492cc02aeb2fed7fc. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 710.440663] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 43fbb2e2-b827-4fc0-aff4-886a26f4550e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 710.441163] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.441376] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.441591] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.449835] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 43fbb2e2-b827-4fc0-aff4-886a26f4550e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 710.452013] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.452013] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.452013] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 710.471380] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 710.471651] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 710.471824] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.147017] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Took 0.12 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 712.162909] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.163387] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.163654] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.203025] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.203025] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.203025] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.203025] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.203025] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.203025] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.210560] nova-conductor[52800]: DEBUG nova.quota [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Getting quotas for project a7f572f853ca47cd9a40604f8a7f6c36. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 712.213285] nova-conductor[52800]: DEBUG nova.quota [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Getting quotas for user dc8d455debe94abf852a3465d733d828 and project a7f572f853ca47cd9a40604f8a7f6c36. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 712.219509] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: b01c69b3-eec6-4577-8288-d4602da9e251] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 712.220200] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.220503] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.220879] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.223844] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: b01c69b3-eec6-4577-8288-d4602da9e251] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 712.224668] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.225073] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.225340] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.240532] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 712.240923] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 712.241225] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 712.609986] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 9bee98ef-48b4-47e6-8afb-e535e58e50cb] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 00b78937-7d5e-4965-91a5-30f8f3c29b85, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9bee98ef-48b4-47e6-8afb-e535e58e50cb was re-scheduled: Binding failed for port 00b78937-7d5e-4965-91a5-30f8f3c29b85, please check neutron logs for more information.\n'] [ 712.611674] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 712.611674] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9bee98ef-48b4-47e6-8afb-e535e58e50cb.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9bee98ef-48b4-47e6-8afb-e535e58e50cb. [ 712.611674] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 9bee98ef-48b4-47e6-8afb-e535e58e50cb] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9bee98ef-48b4-47e6-8afb-e535e58e50cb. [ 712.652944] nova-conductor[52799]: DEBUG nova.network.neutron [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 9bee98ef-48b4-47e6-8afb-e535e58e50cb] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 712.732150] nova-conductor[52799]: DEBUG nova.network.neutron [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 9bee98ef-48b4-47e6-8afb-e535e58e50cb] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 712.739879] nova-conductor[52799]: DEBUG nova.network.neutron [None req-616d50c4-67f7-421a-9f90-205c09c00959 tempest-ServersTestMultiNic-1861207233 tempest-ServersTestMultiNic-1861207233-project-member] [instance: 9bee98ef-48b4-47e6-8afb-e535e58e50cb] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 713.749635] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] [instance: 9e88b24c-500d-4efb-8563-093dd4d0378d] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9c7bd52d-273d-4d17-8e75-b836df862857, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9e88b24c-500d-4efb-8563-093dd4d0378d was re-scheduled: Binding failed for port 9c7bd52d-273d-4d17-8e75-b836df862857, please check neutron logs for more information.\n'] [ 713.750228] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 713.750449] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9e88b24c-500d-4efb-8563-093dd4d0378d.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9e88b24c-500d-4efb-8563-093dd4d0378d. [ 713.750681] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] [instance: 9e88b24c-500d-4efb-8563-093dd4d0378d] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9e88b24c-500d-4efb-8563-093dd4d0378d. [ 713.775966] nova-conductor[52799]: DEBUG nova.network.neutron [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] [instance: 9e88b24c-500d-4efb-8563-093dd4d0378d] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 713.846498] nova-conductor[52799]: DEBUG nova.network.neutron [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] [instance: 9e88b24c-500d-4efb-8563-093dd4d0378d] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 713.853287] nova-conductor[52799]: DEBUG nova.network.neutron [None req-c2451075-b3e5-46a7-bba0-62ebd8fee8f6 tempest-ServerMetadataNegativeTestJSON-885798172 tempest-ServerMetadataNegativeTestJSON-885798172-project-member] [instance: 9e88b24c-500d-4efb-8563-093dd4d0378d] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 714.190831] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 714.210874] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.211119] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.211397] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.254395] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.254989] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.254989] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.259944] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.259944] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.259944] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.271744] nova-conductor[52799]: DEBUG nova.quota [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Getting quotas for project 33b22598eb074c35b7782df547b1cdea. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 714.274952] nova-conductor[52799]: DEBUG nova.quota [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Getting quotas for user dd73536207f046218914901213c53c5c and project 33b22598eb074c35b7782df547b1cdea. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 714.283427] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] [instance: fc85402b-7fcc-4060-b16a-f82d70d6886b] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 714.283745] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.283956] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.284137] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.289918] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] [instance: fc85402b-7fcc-4060-b16a-f82d70d6886b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 714.290593] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.290686] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.290841] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.340256] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.340554] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.340683] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.867386] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 714.881364] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.881691] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.881884] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.915504] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.915602] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.915734] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.916099] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.916286] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.916446] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.927506] nova-conductor[52800]: DEBUG nova.quota [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Getting quotas for project 65241354dfa84a61977e9f11a0483dc9. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 714.931898] nova-conductor[52800]: DEBUG nova.quota [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Getting quotas for user e61100842e42452c920d522726703641 and project 65241354dfa84a61977e9f11a0483dc9. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 714.938918] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: e75afc9c-035c-4926-b72a-d570b5f2e6f0] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 714.940166] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.940166] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.940166] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.942979] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: e75afc9c-035c-4926-b72a-d570b5f2e6f0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 714.944100] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.944100] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.944100] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 714.964072] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 714.964576] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 714.964576] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 716.048584] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: c7cb800a-3634-44e4-bb18-fab9d2e86c7e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port d075e52f-7ad7-43b6-8bcc-5b50c7a4ec75, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c7cb800a-3634-44e4-bb18-fab9d2e86c7e was re-scheduled: Binding failed for port d075e52f-7ad7-43b6-8bcc-5b50c7a4ec75, please check neutron logs for more information.\n'] [ 716.048584] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 716.048584] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c7cb800a-3634-44e4-bb18-fab9d2e86c7e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c7cb800a-3634-44e4-bb18-fab9d2e86c7e. [ 716.048584] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: c7cb800a-3634-44e4-bb18-fab9d2e86c7e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c7cb800a-3634-44e4-bb18-fab9d2e86c7e. [ 716.083437] nova-conductor[52800]: DEBUG nova.network.neutron [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: c7cb800a-3634-44e4-bb18-fab9d2e86c7e] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 716.179668] nova-conductor[52800]: DEBUG nova.network.neutron [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: c7cb800a-3634-44e4-bb18-fab9d2e86c7e] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 716.183106] nova-conductor[52800]: DEBUG nova.network.neutron [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: c7cb800a-3634-44e4-bb18-fab9d2e86c7e] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 717.774173] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 717.788152] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 717.788553] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 717.788809] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 717.819946] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 717.820461] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 717.820805] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 717.821372] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 717.821624] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 717.821831] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 717.833279] nova-conductor[52800]: DEBUG nova.quota [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Getting quotas for project 65241354dfa84a61977e9f11a0483dc9. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 717.835954] nova-conductor[52800]: DEBUG nova.quota [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Getting quotas for user e61100842e42452c920d522726703641 and project 65241354dfa84a61977e9f11a0483dc9. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 717.842448] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: 47d86b97-4bf1-40d4-b666-a530901d28dd] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 717.844033] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 717.844033] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 717.844033] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 717.846175] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: 47d86b97-4bf1-40d4-b666-a530901d28dd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 717.846859] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 717.847103] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 717.847293] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 717.865051] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 717.865267] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 717.865429] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 718.852914] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 8fcf260d-2796-4972-b217-95954e309a6e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0e04edf6-7d03-4368-98ac-203be2fde2ed, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 8fcf260d-2796-4972-b217-95954e309a6e was re-scheduled: Binding failed for port 0e04edf6-7d03-4368-98ac-203be2fde2ed, please check neutron logs for more information.\n'] [ 718.855113] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 718.855113] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8fcf260d-2796-4972-b217-95954e309a6e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8fcf260d-2796-4972-b217-95954e309a6e. [ 718.855113] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 8fcf260d-2796-4972-b217-95954e309a6e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 8fcf260d-2796-4972-b217-95954e309a6e. [ 718.880803] nova-conductor[52799]: DEBUG nova.network.neutron [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 8fcf260d-2796-4972-b217-95954e309a6e] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 718.996193] nova-conductor[52799]: DEBUG nova.network.neutron [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 8fcf260d-2796-4972-b217-95954e309a6e] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 719.008777] nova-conductor[52799]: DEBUG nova.network.neutron [None req-0e92c309-cbac-45de-8e98-2aaf7268129d tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 8fcf260d-2796-4972-b217-95954e309a6e] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 720.528423] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c9c2d371-978e-4037-ba78-9b44f40765bd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 6e276e45-fe27-414d-ba81-a3de27e5773a, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c9c2d371-978e-4037-ba78-9b44f40765bd was re-scheduled: Binding failed for port 6e276e45-fe27-414d-ba81-a3de27e5773a, please check neutron logs for more information.\n'] [ 720.529074] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 720.529320] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9c2d371-978e-4037-ba78-9b44f40765bd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9c2d371-978e-4037-ba78-9b44f40765bd. [ 720.529659] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c9c2d371-978e-4037-ba78-9b44f40765bd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c9c2d371-978e-4037-ba78-9b44f40765bd. [ 720.556755] nova-conductor[52800]: DEBUG nova.network.neutron [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c9c2d371-978e-4037-ba78-9b44f40765bd] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 720.653492] nova-conductor[52800]: DEBUG nova.network.neutron [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c9c2d371-978e-4037-ba78-9b44f40765bd] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 720.660397] nova-conductor[52800]: DEBUG nova.network.neutron [None req-c6361768-afea-4756-bbb0-dabacec97540 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c9c2d371-978e-4037-ba78-9b44f40765bd] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 721.467488] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: 156cf534-81ca-4cc6-9b0d-2d245016c53c] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 22a86f22-b09e-42d4-94fe-94f6c03a4a0b, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 156cf534-81ca-4cc6-9b0d-2d245016c53c was re-scheduled: Binding failed for port 22a86f22-b09e-42d4-94fe-94f6c03a4a0b, please check neutron logs for more information.\n'] [ 721.469797] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 721.469797] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 156cf534-81ca-4cc6-9b0d-2d245016c53c.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 156cf534-81ca-4cc6-9b0d-2d245016c53c. [ 721.469797] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: 156cf534-81ca-4cc6-9b0d-2d245016c53c] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 156cf534-81ca-4cc6-9b0d-2d245016c53c. [ 721.501352] nova-conductor[52799]: DEBUG nova.network.neutron [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: 156cf534-81ca-4cc6-9b0d-2d245016c53c] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 721.566569] nova-conductor[52799]: DEBUG nova.network.neutron [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: 156cf534-81ca-4cc6-9b0d-2d245016c53c] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 721.580959] nova-conductor[52799]: DEBUG nova.network.neutron [None req-3856ad9c-4c98-4f9f-9fa6-f556526fd8a6 tempest-ServerRescueNegativeTestJSON-1532022264 tempest-ServerRescueNegativeTestJSON-1532022264-project-member] [instance: 156cf534-81ca-4cc6-9b0d-2d245016c53c] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 723.077919] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Took 0.14 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 723.118565] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.118805] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.118977] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.182933] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.184194] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.184194] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.184194] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.184194] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.184194] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.194419] nova-conductor[52800]: DEBUG nova.quota [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Getting quotas for project 0be6718d0cbe4351a06b59576311c7f8. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 723.198235] nova-conductor[52800]: DEBUG nova.quota [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Getting quotas for user 56bb638542d440639e1a38b10e80fb1e and project 0be6718d0cbe4351a06b59576311c7f8. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 723.209151] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 9614d3ee-0911-4b50-9875-93ef3f7f2b5f] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 723.209151] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.209151] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.209151] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.213343] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 9614d3ee-0911-4b50-9875-93ef3f7f2b5f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 723.214031] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.214244] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.215377] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.232728] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.232960] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.233567] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.427606] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Took 0.15 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 723.452191] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.452191] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.452191] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.509346] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.509682] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.509860] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.510258] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.510442] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.510649] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.523396] nova-conductor[52799]: DEBUG nova.quota [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Getting quotas for project b00aed0dfc1040dab445131a92d0ef27. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 723.526411] nova-conductor[52799]: DEBUG nova.quota [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Getting quotas for user d7aaa8ce958041a48a24e19ccdc295b4 and project b00aed0dfc1040dab445131a92d0ef27. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 723.538910] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] [instance: ab6859e4-807d-4b5f-943b-6491ed211c75] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 723.539468] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.539705] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.539867] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.543460] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] [instance: ab6859e4-807d-4b5f-943b-6491ed211c75] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 723.544721] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.544721] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.544874] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 723.561367] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 723.561586] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 723.561762] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 724.345472] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 21decdcc-d46e-4851-8eee-8a89912b5691, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b was re-scheduled: Binding failed for port 21decdcc-d46e-4851-8eee-8a89912b5691, please check neutron logs for more information.\n'] [ 724.346201] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 724.346828] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b. [ 724.346944] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b. [ 724.384087] nova-conductor[52800]: DEBUG nova.network.neutron [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 724.451039] nova-conductor[52800]: DEBUG nova.network.neutron [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 724.454748] nova-conductor[52800]: DEBUG nova.network.neutron [None req-7e828195-3163-4e64-9ad9-2ccd9055d3d5 tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: c3e9f1b6-5feb-4d0a-ac70-67918b66fb0b] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 726.019822] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 43fbb2e2-b827-4fc0-aff4-886a26f4550e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 3b7f356e-c9d5-4d4a-b070-7f55ba4a3e59, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 43fbb2e2-b827-4fc0-aff4-886a26f4550e was re-scheduled: Binding failed for port 3b7f356e-c9d5-4d4a-b070-7f55ba4a3e59, please check neutron logs for more information.\n'] [ 726.020572] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 726.020779] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 43fbb2e2-b827-4fc0-aff4-886a26f4550e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 43fbb2e2-b827-4fc0-aff4-886a26f4550e. [ 726.021101] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 43fbb2e2-b827-4fc0-aff4-886a26f4550e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 43fbb2e2-b827-4fc0-aff4-886a26f4550e. [ 726.054102] nova-conductor[52799]: DEBUG nova.network.neutron [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 43fbb2e2-b827-4fc0-aff4-886a26f4550e] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 726.103063] nova-conductor[52799]: DEBUG nova.network.neutron [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 43fbb2e2-b827-4fc0-aff4-886a26f4550e] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 726.111115] nova-conductor[52799]: DEBUG nova.network.neutron [None req-089040d8-fa96-4a76-be1b-d929bfa36203 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 43fbb2e2-b827-4fc0-aff4-886a26f4550e] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 728.274614] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: b01c69b3-eec6-4577-8288-d4602da9e251] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 9670ba89-f99d-442f-a928-00c5989967cf, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance b01c69b3-eec6-4577-8288-d4602da9e251 was re-scheduled: Binding failed for port 9670ba89-f99d-442f-a928-00c5989967cf, please check neutron logs for more information.\n'] [ 728.276099] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 728.276099] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b01c69b3-eec6-4577-8288-d4602da9e251.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b01c69b3-eec6-4577-8288-d4602da9e251. [ 728.276099] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: b01c69b3-eec6-4577-8288-d4602da9e251] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b01c69b3-eec6-4577-8288-d4602da9e251. [ 728.320701] nova-conductor[52799]: DEBUG nova.network.neutron [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: b01c69b3-eec6-4577-8288-d4602da9e251] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 728.374311] nova-conductor[52799]: DEBUG nova.network.neutron [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: b01c69b3-eec6-4577-8288-d4602da9e251] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 728.381570] nova-conductor[52799]: DEBUG nova.network.neutron [None req-d5dd79ad-47c2-4efe-b1c2-54c3c2bb9651 tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: b01c69b3-eec6-4577-8288-d4602da9e251] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 728.473724] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Took 0.17 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 728.486374] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.487063] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.487251] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.582152] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.582152] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.582152] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.582152] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.582152] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.582152] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.591974] nova-conductor[52799]: DEBUG nova.quota [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Getting quotas for project e05387f01e784108985ef588ab2b8094. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 728.599303] nova-conductor[52799]: DEBUG nova.quota [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Getting quotas for user 9a32a17cf2b0429d97c45e2e0574d14f and project e05387f01e784108985ef588ab2b8094. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 728.609238] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: bd447698-8d52-4576-9d86-1a22e36bc3d5] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 728.609238] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.609238] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.609685] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.617160] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: bd447698-8d52-4576-9d86-1a22e36bc3d5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 728.617859] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.618088] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.618274] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.660836] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 728.660836] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 728.660836] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 728.792567] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port db9c5b4c-7f66-4453-85b1-d47606e0a329, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2 was re-scheduled: Binding failed for port db9c5b4c-7f66-4453-85b1-d47606e0a329, please check neutron logs for more information.\n'] [ 728.793321] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 728.794099] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2. [ 728.794099] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2. [ 728.840481] nova-conductor[52799]: DEBUG nova.network.neutron [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 728.866773] nova-conductor[52799]: DEBUG nova.network.neutron [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 728.874982] nova-conductor[52799]: DEBUG nova.network.neutron [None req-1b405b20-af63-4812-ba02-dc6023967f4b tempest-ListServerFiltersTestJSON-1580368180 tempest-ListServerFiltersTestJSON-1580368180-project-member] [instance: fd4ad598-3fa2-4a7a-9226-9cf9dba03ce2] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 729.938569] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] [instance: fc85402b-7fcc-4060-b16a-f82d70d6886b] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 4c38e9c1-0c1e-4465-acd5-be6ebc735624, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance fc85402b-7fcc-4060-b16a-f82d70d6886b was re-scheduled: Binding failed for port 4c38e9c1-0c1e-4465-acd5-be6ebc735624, please check neutron logs for more information.\n'] [ 729.938569] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 729.938569] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fc85402b-7fcc-4060-b16a-f82d70d6886b.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fc85402b-7fcc-4060-b16a-f82d70d6886b. [ 729.938569] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] [instance: fc85402b-7fcc-4060-b16a-f82d70d6886b] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance fc85402b-7fcc-4060-b16a-f82d70d6886b. [ 729.962392] nova-conductor[52800]: DEBUG nova.network.neutron [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] [instance: fc85402b-7fcc-4060-b16a-f82d70d6886b] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 730.013771] nova-conductor[52800]: DEBUG nova.network.neutron [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] [instance: fc85402b-7fcc-4060-b16a-f82d70d6886b] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 730.018309] nova-conductor[52800]: DEBUG nova.network.neutron [None req-2c4217d4-e9e0-46ea-830d-e5ef697e4504 tempest-ServerRescueTestJSONUnderV235-724313637 tempest-ServerRescueTestJSONUnderV235-724313637-project-member] [instance: fc85402b-7fcc-4060-b16a-f82d70d6886b] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 730.381459] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: 7466dfd3-8756-40eb-91fd-c87f16b627ef] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 7466dfd3-8756-40eb-91fd-c87f16b627ef was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 730.381459] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 730.382095] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7466dfd3-8756-40eb-91fd-c87f16b627ef.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7466dfd3-8756-40eb-91fd-c87f16b627ef. [ 730.382095] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-436b2d56-1260-4986-ae3c-b0d3c111722e tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: 7466dfd3-8756-40eb-91fd-c87f16b627ef] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 7466dfd3-8756-40eb-91fd-c87f16b627ef. [ 730.389674] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 730.424091] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.424208] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.424404] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.457268] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.457535] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.457716] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.458095] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.458317] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.458430] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.469157] nova-conductor[52800]: DEBUG nova.quota [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Getting quotas for project 2d138ee86d5a4657a3c40323d42a362b. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 730.471387] nova-conductor[52800]: DEBUG nova.quota [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Getting quotas for user bfa6215d0fdf4e45821e1776d7c9e7d0 and project 2d138ee86d5a4657a3c40323d42a362b. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 730.486979] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] [instance: 35cae673-166d-4ffc-90fb-aee3bdfd1710] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 730.489060] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.489060] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.489060] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.492208] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] [instance: 35cae673-166d-4ffc-90fb-aee3bdfd1710] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 730.492536] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.492743] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.493134] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 730.510871] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 730.511107] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 730.512437] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.394037] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Took 0.16 seconds to select destinations for 1 instance(s). {{(pid=52799) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 731.403627] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.403843] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.404017] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.433143] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.433346] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.433512] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.433873] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.434039] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.434193] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.446848] nova-conductor[52799]: DEBUG nova.quota [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Getting quotas for project cc5a77f597ba4db3a043b8962c824544. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 731.449775] nova-conductor[52799]: DEBUG nova.quota [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Getting quotas for user 5d6a5236d3ea429bab0088e848dd4599 and project cc5a77f597ba4db3a043b8962c824544. Resources: {'cores', 'instances', 'ram'} {{(pid=52799) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 731.456460] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] [instance: 946c73f8-1ed8-4180-a9d7-0b2970c4367e] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52799) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 731.457149] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.457406] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.457546] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.463029] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] [instance: 946c73f8-1ed8-4180-a9d7-0b2970c4367e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 731.463029] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.463029] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.463029] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.477125] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.478877] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.478877] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.596100] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Took 0.24 seconds to select destinations for 2 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 731.615514] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.615760] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.615930] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.647058] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.647283] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.647448] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.691022] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.691256] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.691425] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.691778] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.691959] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.692516] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.701290] nova-conductor[52800]: DEBUG nova.quota [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Getting quotas for project a98b1fd8031545e381db0682e508fc18. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 731.703682] nova-conductor[52800]: DEBUG nova.quota [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Getting quotas for user 4cc2b0ed84534852a16f9fdd4a8977f7 and project a98b1fd8031545e381db0682e508fc18. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 731.710521] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 73b58043-e025-48ff-a22a-4d226c545456] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 731.710800] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.710999] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.711380] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.717105] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 73b58043-e025-48ff-a22a-4d226c545456] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 731.717905] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.718123] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.718450] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.732890] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.733119] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.733289] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.742899] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 031e33fa-92ab-483b-ab38-ecf3bbfd1374] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 731.742899] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.742899] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.742899] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.747682] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 031e33fa-92ab-483b-ab38-ecf3bbfd1374] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 731.748430] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.748569] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.749033] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 731.764262] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 731.764475] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 731.764633] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 732.052140] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: 47d86b97-4bf1-40d4-b666-a530901d28dd] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 40443b08-0a04-4ae9-9f18-468b0fb8d3e6, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 47d86b97-4bf1-40d4-b666-a530901d28dd was re-scheduled: Binding failed for port 40443b08-0a04-4ae9-9f18-468b0fb8d3e6, please check neutron logs for more information.\n'] [ 732.052751] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 732.053066] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 47d86b97-4bf1-40d4-b666-a530901d28dd.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 47d86b97-4bf1-40d4-b666-a530901d28dd. [ 732.053396] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: 47d86b97-4bf1-40d4-b666-a530901d28dd] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 47d86b97-4bf1-40d4-b666-a530901d28dd. [ 732.075623] nova-conductor[52800]: DEBUG nova.network.neutron [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: 47d86b97-4bf1-40d4-b666-a530901d28dd] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 732.148274] nova-conductor[52800]: DEBUG nova.network.neutron [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: 47d86b97-4bf1-40d4-b666-a530901d28dd] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 732.155583] nova-conductor[52800]: DEBUG nova.network.neutron [None req-351813c6-aeba-4a5e-822b-065981327928 tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: 47d86b97-4bf1-40d4-b666-a530901d28dd] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 733.588568] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Took 0.18 seconds to select destinations for 1 instance(s). {{(pid=52800) _schedule_instances /opt/stack/nova/nova/conductor/manager.py:943}} [ 733.606209] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 733.606425] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 733.606594] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 733.650984] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 733.651224] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 733.651387] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 733.651982] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 733.652199] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 733.652358] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 733.663166] nova-conductor[52800]: DEBUG nova.quota [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Getting quotas for project bd9480d686364ebab0d87bd617f302fa. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:393}} [ 733.665649] nova-conductor[52800]: DEBUG nova.quota [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Getting quotas for user 6e12895c63134d80bf3f4f545baf2554 and project bd9480d686364ebab0d87bd617f302fa. Resources: {'cores', 'instances', 'ram'} {{(pid=52800) _get_quotas /opt/stack/nova/nova/quota.py:383}} [ 733.675213] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] [instance: ad70ab2b-17e2-4cf1-9411-272aec5bfb8a] Selected host: cpu-1; Selected node: domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28; Alternates: [] {{(pid=52800) schedule_and_build_instances /opt/stack/nova/nova/conductor/manager.py:1713}} [ 733.675652] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 733.675847] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 733.676027] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 733.679178] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] [instance: ad70ab2b-17e2-4cf1-9411-272aec5bfb8a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 733.679817] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 733.680027] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 733.680192] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 733.718980] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 733.719222] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 733.719387] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 734.284417] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: e75afc9c-035c-4926-b72a-d570b5f2e6f0] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port ca937adf-d12a-4397-bdb1-e9c32bd7d7a4, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance e75afc9c-035c-4926-b72a-d570b5f2e6f0 was re-scheduled: Binding failed for port ca937adf-d12a-4397-bdb1-e9c32bd7d7a4, please check neutron logs for more information.\n'] [ 734.285111] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 734.285390] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e75afc9c-035c-4926-b72a-d570b5f2e6f0.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e75afc9c-035c-4926-b72a-d570b5f2e6f0. [ 734.286142] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: e75afc9c-035c-4926-b72a-d570b5f2e6f0] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance e75afc9c-035c-4926-b72a-d570b5f2e6f0. [ 734.317128] nova-conductor[52799]: DEBUG nova.network.neutron [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: e75afc9c-035c-4926-b72a-d570b5f2e6f0] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 734.411114] nova-conductor[52799]: DEBUG nova.network.neutron [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: e75afc9c-035c-4926-b72a-d570b5f2e6f0] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 734.414911] nova-conductor[52799]: DEBUG nova.network.neutron [None req-52da69a3-269c-4578-bdbd-d3c315a7f06e tempest-ServersAdminTestJSON-1751613823 tempest-ServersAdminTestJSON-1751613823-project-member] [instance: e75afc9c-035c-4926-b72a-d570b5f2e6f0] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 735.052932] nova-conductor[52799]: Traceback (most recent call last): [ 735.052932] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 735.052932] nova-conductor[52799]: return func(*args, **kwargs) [ 735.052932] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 735.052932] nova-conductor[52799]: selections = self._select_destinations( [ 735.052932] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 735.052932] nova-conductor[52799]: selections = self._schedule( [ 735.052932] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 735.052932] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 735.052932] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 735.052932] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 735.052932] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.052932] nova-conductor[52799]: ERROR nova.conductor.manager [ 735.060384] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 735.060603] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 735.060776] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 735.103658] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 1997b816-146a-460b-b55d-664e5e228759] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 735.105411] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 735.105705] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 735.105888] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 735.109664] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 735.109664] nova-conductor[52799]: Traceback (most recent call last): [ 735.109664] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 735.109664] nova-conductor[52799]: return func(*args, **kwargs) [ 735.109664] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 735.109664] nova-conductor[52799]: selections = self._select_destinations( [ 735.109664] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 735.109664] nova-conductor[52799]: selections = self._schedule( [ 735.109664] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 735.109664] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 735.109664] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 735.109664] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 735.109664] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 735.109664] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 735.110666] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-2516b778-14fe-4c02-8d78-70287e90db00 tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: 1997b816-146a-460b-b55d-664e5e228759] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 735.434716] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 9614d3ee-0911-4b50-9875-93ef3f7f2b5f] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 54daa46a-e3a4-4c3a-80ab-96f92c47ae45, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 9614d3ee-0911-4b50-9875-93ef3f7f2b5f was re-scheduled: Binding failed for port 54daa46a-e3a4-4c3a-80ab-96f92c47ae45, please check neutron logs for more information.\n'] [ 735.435965] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 735.436526] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9614d3ee-0911-4b50-9875-93ef3f7f2b5f.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9614d3ee-0911-4b50-9875-93ef3f7f2b5f. [ 735.436913] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 9614d3ee-0911-4b50-9875-93ef3f7f2b5f] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 9614d3ee-0911-4b50-9875-93ef3f7f2b5f. [ 735.476116] nova-conductor[52799]: DEBUG nova.network.neutron [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 9614d3ee-0911-4b50-9875-93ef3f7f2b5f] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 735.603742] nova-conductor[52799]: DEBUG nova.network.neutron [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 9614d3ee-0911-4b50-9875-93ef3f7f2b5f] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 735.606081] nova-conductor[52799]: DEBUG nova.network.neutron [None req-9d85b446-e533-4385-85de-1ac0d0f19f0b tempest-AttachInterfacesTestJSON-1051727072 tempest-AttachInterfacesTestJSON-1051727072-project-member] [instance: 9614d3ee-0911-4b50-9875-93ef3f7f2b5f] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 741.610142] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: bd447698-8d52-4576-9d86-1a22e36bc3d5] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 658e7dd4-2de9-447d-8402-096e9544e744, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance bd447698-8d52-4576-9d86-1a22e36bc3d5 was re-scheduled: Binding failed for port 658e7dd4-2de9-447d-8402-096e9544e744, please check neutron logs for more information.\n'] [ 741.611770] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 741.611770] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bd447698-8d52-4576-9d86-1a22e36bc3d5.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bd447698-8d52-4576-9d86-1a22e36bc3d5. [ 741.611770] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: bd447698-8d52-4576-9d86-1a22e36bc3d5] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance bd447698-8d52-4576-9d86-1a22e36bc3d5. [ 741.640189] nova-conductor[52800]: DEBUG nova.network.neutron [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: bd447698-8d52-4576-9d86-1a22e36bc3d5] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 741.749809] nova-conductor[52800]: DEBUG nova.network.neutron [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: bd447698-8d52-4576-9d86-1a22e36bc3d5] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 741.761015] nova-conductor[52800]: DEBUG nova.network.neutron [None req-37b5320c-8cb9-46d1-9324-af075ff3ca1d tempest-SecurityGroupsTestJSON-1433999459 tempest-SecurityGroupsTestJSON-1433999459-project-member] [instance: bd447698-8d52-4576-9d86-1a22e36bc3d5] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 741.858948] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] [instance: ab6859e4-807d-4b5f-943b-6491ed211c75] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 13ca33c8-9dde-4869-a11f-1bd3910b59be, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ab6859e4-807d-4b5f-943b-6491ed211c75 was re-scheduled: Binding failed for port 13ca33c8-9dde-4869-a11f-1bd3910b59be, please check neutron logs for more information.\n'] [ 741.859468] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 741.859749] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ab6859e4-807d-4b5f-943b-6491ed211c75.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ab6859e4-807d-4b5f-943b-6491ed211c75. [ 741.859973] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] [instance: ab6859e4-807d-4b5f-943b-6491ed211c75] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ab6859e4-807d-4b5f-943b-6491ed211c75. [ 741.880171] nova-conductor[52800]: DEBUG nova.network.neutron [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] [instance: ab6859e4-807d-4b5f-943b-6491ed211c75] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 741.971153] nova-conductor[52800]: DEBUG nova.network.neutron [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] [instance: ab6859e4-807d-4b5f-943b-6491ed211c75] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 741.976155] nova-conductor[52800]: DEBUG nova.network.neutron [None req-84e10c45-1787-4e24-9273-ffcb4e8fc8ea tempest-ServerActionsTestOtherA-1143455132 tempest-ServerActionsTestOtherA-1143455132-project-member] [instance: ab6859e4-807d-4b5f-943b-6491ed211c75] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 742.397835] nova-conductor[52799]: Traceback (most recent call last): [ 742.397835] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 742.397835] nova-conductor[52799]: return func(*args, **kwargs) [ 742.397835] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 742.397835] nova-conductor[52799]: selections = self._select_destinations( [ 742.397835] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 742.397835] nova-conductor[52799]: selections = self._schedule( [ 742.397835] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 742.397835] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 742.397835] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 742.397835] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 742.397835] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.397835] nova-conductor[52799]: ERROR nova.conductor.manager [ 742.404647] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.404865] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.405045] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.468932] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: f064209e-cc0b-4843-9b48-0ddf6cd80317] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 742.469193] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 742.469354] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 742.469528] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 742.480634] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 742.480634] nova-conductor[52799]: Traceback (most recent call last): [ 742.480634] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 742.480634] nova-conductor[52799]: return func(*args, **kwargs) [ 742.480634] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 742.480634] nova-conductor[52799]: selections = self._select_destinations( [ 742.480634] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 742.480634] nova-conductor[52799]: selections = self._schedule( [ 742.480634] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 742.480634] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 742.480634] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 742.480634] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 742.480634] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 742.480634] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 742.481109] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-9ca38bf9-0a97-4e59-be1c-356eeb54f25e tempest-ServerDiskConfigTestJSON-249548930 tempest-ServerDiskConfigTestJSON-249548930-project-member] [instance: f064209e-cc0b-4843-9b48-0ddf6cd80317] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 744.774612] nova-conductor[52799]: Traceback (most recent call last): [ 744.774612] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 744.774612] nova-conductor[52799]: return func(*args, **kwargs) [ 744.774612] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 744.774612] nova-conductor[52799]: selections = self._select_destinations( [ 744.774612] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 744.774612] nova-conductor[52799]: selections = self._schedule( [ 744.774612] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 744.774612] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 744.774612] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 744.774612] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 744.774612] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.774612] nova-conductor[52799]: ERROR nova.conductor.manager [ 744.793123] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 744.793958] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 744.793958] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 744.870381] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] [instance: 35cae673-166d-4ffc-90fb-aee3bdfd1710] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 97ab4f54-0a33-4866-8e90-3302cbbff541, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 35cae673-166d-4ffc-90fb-aee3bdfd1710 was re-scheduled: Binding failed for port 97ab4f54-0a33-4866-8e90-3302cbbff541, please check neutron logs for more information.\n'] [ 744.870782] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 744.870868] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 35cae673-166d-4ffc-90fb-aee3bdfd1710.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 35cae673-166d-4ffc-90fb-aee3bdfd1710. [ 744.871116] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] [instance: 35cae673-166d-4ffc-90fb-aee3bdfd1710] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 35cae673-166d-4ffc-90fb-aee3bdfd1710. [ 744.874119] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: a5ccea22-a4ca-4884-9efe-9d630038df57] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 744.874836] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 744.875088] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 744.875215] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 744.879857] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 744.879857] nova-conductor[52799]: Traceback (most recent call last): [ 744.879857] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 744.879857] nova-conductor[52799]: return func(*args, **kwargs) [ 744.879857] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 744.879857] nova-conductor[52799]: selections = self._select_destinations( [ 744.879857] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 744.879857] nova-conductor[52799]: selections = self._schedule( [ 744.879857] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 744.879857] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 744.879857] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 744.879857] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 744.879857] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 744.879857] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 744.880407] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-549a115c-34af-469d-b82c-f2db27daea6d tempest-AttachVolumeTestJSON-1745626859 tempest-AttachVolumeTestJSON-1745626859-project-member] [instance: a5ccea22-a4ca-4884-9efe-9d630038df57] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 744.898063] nova-conductor[52800]: DEBUG nova.network.neutron [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] [instance: 35cae673-166d-4ffc-90fb-aee3bdfd1710] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 744.979040] nova-conductor[52800]: DEBUG nova.network.neutron [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] [instance: 35cae673-166d-4ffc-90fb-aee3bdfd1710] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 744.982575] nova-conductor[52800]: DEBUG nova.network.neutron [None req-4b034ead-b341-4094-8a86-1293a94d7995 tempest-ImagesOneServerTestJSON-356266125 tempest-ImagesOneServerTestJSON-356266125-project-member] [instance: 35cae673-166d-4ffc-90fb-aee3bdfd1710] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 745.712834] nova-conductor[52800]: Traceback (most recent call last): [ 745.712834] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 745.712834] nova-conductor[52800]: return func(*args, **kwargs) [ 745.712834] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 745.712834] nova-conductor[52800]: selections = self._select_destinations( [ 745.712834] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 745.712834] nova-conductor[52800]: selections = self._schedule( [ 745.712834] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 745.712834] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 745.712834] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 745.712834] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 745.712834] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.712834] nova-conductor[52800]: ERROR nova.conductor.manager [ 745.724887] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 745.724887] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 745.724887] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 745.779086] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 3b4d70cf-f984-4c00-8882-a1e8f7b5f248] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 745.779974] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 745.780107] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 745.780286] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 745.784892] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 745.784892] nova-conductor[52800]: Traceback (most recent call last): [ 745.784892] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 745.784892] nova-conductor[52800]: return func(*args, **kwargs) [ 745.784892] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 745.784892] nova-conductor[52800]: selections = self._select_destinations( [ 745.784892] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 745.784892] nova-conductor[52800]: selections = self._schedule( [ 745.784892] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 745.784892] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 745.784892] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 745.784892] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 745.784892] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 745.784892] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 745.788424] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-a7bb380c-abec-4fbe-b0e4-15b7317abc5b tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 3b4d70cf-f984-4c00-8882-a1e8f7b5f248] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 746.395137] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 031e33fa-92ab-483b-ab38-ecf3bbfd1374] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 521c92d4-0858-4e8f-a8a6-4774cc3624ae, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 031e33fa-92ab-483b-ab38-ecf3bbfd1374 was re-scheduled: Binding failed for port 521c92d4-0858-4e8f-a8a6-4774cc3624ae, please check neutron logs for more information.\n'] [ 746.396075] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 746.396075] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 031e33fa-92ab-483b-ab38-ecf3bbfd1374.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 031e33fa-92ab-483b-ab38-ecf3bbfd1374. [ 746.396239] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 031e33fa-92ab-483b-ab38-ecf3bbfd1374] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 031e33fa-92ab-483b-ab38-ecf3bbfd1374. [ 746.423627] nova-conductor[52800]: DEBUG nova.network.neutron [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 031e33fa-92ab-483b-ab38-ecf3bbfd1374] deallocate_for_instance() {{(pid=52800) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 746.475105] nova-conductor[52800]: DEBUG nova.network.neutron [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 031e33fa-92ab-483b-ab38-ecf3bbfd1374] Instance cache missing network info. {{(pid=52800) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 746.479294] nova-conductor[52800]: DEBUG nova.network.neutron [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 031e33fa-92ab-483b-ab38-ecf3bbfd1374] Updating instance_info_cache with network_info: [] {{(pid=52800) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 746.716315] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 73b58043-e025-48ff-a22a-4d226c545456] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 287f3ea7-5f8f-462d-b0a4-2bb5a46b7eed, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance 73b58043-e025-48ff-a22a-4d226c545456 was re-scheduled: Binding failed for port 287f3ea7-5f8f-462d-b0a4-2bb5a46b7eed, please check neutron logs for more information.\n'] [ 746.717406] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 746.717781] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 73b58043-e025-48ff-a22a-4d226c545456.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 73b58043-e025-48ff-a22a-4d226c545456. [ 746.718172] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 73b58043-e025-48ff-a22a-4d226c545456] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 73b58043-e025-48ff-a22a-4d226c545456. [ 746.741123] nova-conductor[52799]: DEBUG nova.network.neutron [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 73b58043-e025-48ff-a22a-4d226c545456] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 746.977601] nova-conductor[52799]: DEBUG nova.network.neutron [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 73b58043-e025-48ff-a22a-4d226c545456] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 746.981317] nova-conductor[52799]: DEBUG nova.network.neutron [None req-5f5fea44-936f-46e8-9776-aee45b3037a5 tempest-MultipleCreateTestJSON-1686950820 tempest-MultipleCreateTestJSON-1686950820-project-member] [instance: 73b58043-e025-48ff-a22a-4d226c545456] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 748.225372] nova-conductor[52799]: Traceback (most recent call last): [ 748.225372] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 748.225372] nova-conductor[52799]: return func(*args, **kwargs) [ 748.225372] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 748.225372] nova-conductor[52799]: selections = self._select_destinations( [ 748.225372] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 748.225372] nova-conductor[52799]: selections = self._schedule( [ 748.225372] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 748.225372] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 748.225372] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 748.225372] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 748.225372] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.225372] nova-conductor[52799]: ERROR nova.conductor.manager [ 748.243020] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 748.243020] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 748.243020] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 748.340251] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] [instance: ca375850-3a4b-4c6f-8ec8-5708dc2f936e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 748.341017] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 748.341279] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 748.341455] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 748.345548] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 748.345548] nova-conductor[52799]: Traceback (most recent call last): [ 748.345548] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 748.345548] nova-conductor[52799]: return func(*args, **kwargs) [ 748.345548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 748.345548] nova-conductor[52799]: selections = self._select_destinations( [ 748.345548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 748.345548] nova-conductor[52799]: selections = self._schedule( [ 748.345548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 748.345548] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 748.345548] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 748.345548] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 748.345548] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 748.345548] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 748.346151] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-2c29bfab-0ad7-4028-9256-a17fbe46754d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] [instance: ca375850-3a4b-4c6f-8ec8-5708dc2f936e] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 748.665396] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] [instance: ad70ab2b-17e2-4cf1-9411-272aec5bfb8a] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 753, in spawn\n vm_ref = self.build_virtual_machine(instance,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 275, in build_virtual_machine\n vif_infos = vmwarevif.get_vif_info(self._session,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vif.py", line 119, in get_vif_info\n for vif in network_info:\n', ' File "/opt/stack/nova/nova/network/model.py", line 612, in __iter__\n return self._sync_wrapper(fn, *args, **kwargs)\n', ' File "/opt/stack/nova/nova/network/model.py", line 603, in _sync_wrapper\n self.wait()\n', ' File "/opt/stack/nova/nova/network/model.py", line 635, in wait\n self[:] = self._gt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 181, in wait\n return self._exit_event.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/greenthread.py", line 221, in main\n result = function(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/utils.py", line 654, in context_wrapper\n return func(*args, **kwargs)\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1987, in _allocate_network_async\n raise e\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 1965, in _allocate_network_async\n nwinfo = self.network_api.allocate_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1216, in allocate_for_instance\n created_port_ids = self._update_ports_for_instance(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1352, in _update_ports_for_instance\n with excutils.save_and_reraise_exception():\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__\n self.force_reraise()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise\n raise self.value\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 1327, in _update_ports_for_instance\n updated_port = self._update_port(\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 585, in _update_port\n _ensure_no_port_binding_failure(port)\n', ' File "/opt/stack/nova/nova/network/neutron.py", line 294, in _ensure_no_port_binding_failure\n raise exception.PortBindingFailed(port_id=port[\'id\'])\n', 'nova.exception.PortBindingFailed: Binding failed for port 0b909c60-aba4-4d0b-8134-93a9bbbab5da, please check neutron logs for more information.\n', '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', 'nova.exception.RescheduledException: Build of instance ad70ab2b-17e2-4cf1-9411-272aec5bfb8a was re-scheduled: Binding failed for port 0b909c60-aba4-4d0b-8134-93a9bbbab5da, please check neutron logs for more information.\n'] [ 748.665939] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 748.666447] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad70ab2b-17e2-4cf1-9411-272aec5bfb8a.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad70ab2b-17e2-4cf1-9411-272aec5bfb8a. [ 748.666677] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] [instance: ad70ab2b-17e2-4cf1-9411-272aec5bfb8a] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance ad70ab2b-17e2-4cf1-9411-272aec5bfb8a. [ 748.689511] nova-conductor[52799]: DEBUG nova.network.neutron [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] [instance: ad70ab2b-17e2-4cf1-9411-272aec5bfb8a] deallocate_for_instance() {{(pid=52799) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1782}} [ 748.721446] nova-conductor[52799]: DEBUG nova.network.neutron [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] [instance: ad70ab2b-17e2-4cf1-9411-272aec5bfb8a] Instance cache missing network info. {{(pid=52799) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3302}} [ 748.725879] nova-conductor[52799]: DEBUG nova.network.neutron [None req-04286880-fda9-4f78-826e-74ba0d7d7596 tempest-ServersNegativeTestJSON-589792489 tempest-ServersNegativeTestJSON-589792489-project-member] [instance: ad70ab2b-17e2-4cf1-9411-272aec5bfb8a] Updating instance_info_cache with network_info: [] {{(pid=52799) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 750.347525] nova-conductor[52799]: Traceback (most recent call last): [ 750.347525] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 750.347525] nova-conductor[52799]: return func(*args, **kwargs) [ 750.347525] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 750.347525] nova-conductor[52799]: selections = self._select_destinations( [ 750.347525] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 750.347525] nova-conductor[52799]: selections = self._schedule( [ 750.347525] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 750.347525] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 750.347525] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 750.347525] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 750.347525] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.347525] nova-conductor[52799]: ERROR nova.conductor.manager [ 750.359714] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 750.359714] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 750.359714] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 750.416524] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] [instance: 60441106-aaf9-49b2-b146-8b0c764b6016] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 750.417018] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 750.417188] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 750.417358] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 750.421590] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 750.421590] nova-conductor[52799]: Traceback (most recent call last): [ 750.421590] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 750.421590] nova-conductor[52799]: return func(*args, **kwargs) [ 750.421590] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 750.421590] nova-conductor[52799]: selections = self._select_destinations( [ 750.421590] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 750.421590] nova-conductor[52799]: selections = self._schedule( [ 750.421590] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 750.421590] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 750.421590] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 750.421590] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 750.421590] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 750.421590] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 750.421590] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-71f86aa9-a601-4d34-a118-3ef725cfd576 tempest-ServerAddressesNegativeTestJSON-1979938816 tempest-ServerAddressesNegativeTestJSON-1979938816-project-member] [instance: 60441106-aaf9-49b2-b146-8b0c764b6016] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 750.479046] nova-conductor[52800]: Traceback (most recent call last): [ 750.479046] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 750.479046] nova-conductor[52800]: return func(*args, **kwargs) [ 750.479046] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 750.479046] nova-conductor[52800]: selections = self._select_destinations( [ 750.479046] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 750.479046] nova-conductor[52800]: selections = self._schedule( [ 750.479046] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 750.479046] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 750.479046] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 750.479046] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 750.479046] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.479046] nova-conductor[52800]: ERROR nova.conductor.manager [ 750.487931] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 750.488169] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 750.488339] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 750.595716] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] [instance: 58ac1c3e-2870-4504-a7b3-202406b6db02] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 750.596256] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 750.596914] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 750.596914] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 750.604754] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 750.604754] nova-conductor[52800]: Traceback (most recent call last): [ 750.604754] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 750.604754] nova-conductor[52800]: return func(*args, **kwargs) [ 750.604754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 750.604754] nova-conductor[52800]: selections = self._select_destinations( [ 750.604754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 750.604754] nova-conductor[52800]: selections = self._schedule( [ 750.604754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 750.604754] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 750.604754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 750.604754] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 750.604754] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 750.604754] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 750.605305] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-4fc6927c-ab91-45c6-a7fe-53fd2a723ecb tempest-AttachInterfacesV270Test-1136141756 tempest-AttachInterfacesV270Test-1136141756-project-member] [instance: 58ac1c3e-2870-4504-a7b3-202406b6db02] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 752.053068] nova-conductor[52799]: Traceback (most recent call last): [ 752.053068] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 752.053068] nova-conductor[52799]: return func(*args, **kwargs) [ 752.053068] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 752.053068] nova-conductor[52799]: selections = self._select_destinations( [ 752.053068] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 752.053068] nova-conductor[52799]: selections = self._schedule( [ 752.053068] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 752.053068] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 752.053068] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 752.053068] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 752.053068] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.053068] nova-conductor[52799]: ERROR nova.conductor.manager [ 752.059932] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 752.060368] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 752.060368] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 752.116880] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 280bde49-8fab-4fe2-8518-0bebad720479] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 752.118333] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 752.118572] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 752.118745] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 752.122104] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 752.122104] nova-conductor[52799]: Traceback (most recent call last): [ 752.122104] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 752.122104] nova-conductor[52799]: return func(*args, **kwargs) [ 752.122104] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 752.122104] nova-conductor[52799]: selections = self._select_destinations( [ 752.122104] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 752.122104] nova-conductor[52799]: selections = self._schedule( [ 752.122104] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 752.122104] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 752.122104] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 752.122104] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 752.122104] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 752.122104] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 752.122620] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-7a8d3501-e27a-4e8f-8173-e5852dfd56e5 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 280bde49-8fab-4fe2-8518-0bebad720479] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 755.903724] nova-conductor[52800]: Traceback (most recent call last): [ 755.903724] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 755.903724] nova-conductor[52800]: return func(*args, **kwargs) [ 755.903724] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 755.903724] nova-conductor[52800]: selections = self._select_destinations( [ 755.903724] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 755.903724] nova-conductor[52800]: selections = self._schedule( [ 755.903724] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 755.903724] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 755.903724] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 755.903724] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 755.903724] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.903724] nova-conductor[52800]: ERROR nova.conductor.manager [ 755.913265] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 755.913560] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 755.913773] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 755.962920] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 253188ab-2c7c-4839-8d87-817496e09cb0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 755.963733] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 755.963819] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 755.963987] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 755.968099] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 755.968099] nova-conductor[52800]: Traceback (most recent call last): [ 755.968099] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 755.968099] nova-conductor[52800]: return func(*args, **kwargs) [ 755.968099] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 755.968099] nova-conductor[52800]: selections = self._select_destinations( [ 755.968099] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 755.968099] nova-conductor[52800]: selections = self._schedule( [ 755.968099] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 755.968099] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 755.968099] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 755.968099] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 755.968099] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 755.968099] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 755.968714] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-2436e1d8-3ee8-43e9-8851-84bf114022c4 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 253188ab-2c7c-4839-8d87-817496e09cb0] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 758.529982] nova-conductor[52799]: Traceback (most recent call last): [ 758.529982] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 758.529982] nova-conductor[52799]: return func(*args, **kwargs) [ 758.529982] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 758.529982] nova-conductor[52799]: selections = self._select_destinations( [ 758.529982] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 758.529982] nova-conductor[52799]: selections = self._schedule( [ 758.529982] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 758.529982] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 758.529982] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 758.529982] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 758.529982] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.529982] nova-conductor[52799]: ERROR nova.conductor.manager [ 758.537929] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.538188] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.538360] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.580269] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 23095fb2-cc38-4371-affc-e2613a934917] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 758.581086] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 758.581243] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 758.581414] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 758.584828] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 758.584828] nova-conductor[52799]: Traceback (most recent call last): [ 758.584828] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 758.584828] nova-conductor[52799]: return func(*args, **kwargs) [ 758.584828] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 758.584828] nova-conductor[52799]: selections = self._select_destinations( [ 758.584828] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 758.584828] nova-conductor[52799]: selections = self._schedule( [ 758.584828] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 758.584828] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 758.584828] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 758.584828] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 758.584828] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 758.584828] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 758.585372] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-a9c48531-e278-48c9-8fad-51746e5d234c tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 23095fb2-cc38-4371-affc-e2613a934917] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 759.910649] nova-conductor[52800]: Traceback (most recent call last): [ 759.910649] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 759.910649] nova-conductor[52800]: return func(*args, **kwargs) [ 759.910649] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 759.910649] nova-conductor[52800]: selections = self._select_destinations( [ 759.910649] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 759.910649] nova-conductor[52800]: selections = self._schedule( [ 759.910649] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 759.910649] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 759.910649] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 759.910649] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 759.910649] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.910649] nova-conductor[52800]: ERROR nova.conductor.manager [ 759.919379] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 759.919379] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 759.919379] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 759.986807] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 9aed678d-de26-42bd-93df-996e1b4019eb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 759.987548] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 759.987755] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 759.987927] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 759.993754] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 759.993754] nova-conductor[52800]: Traceback (most recent call last): [ 759.993754] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 759.993754] nova-conductor[52800]: return func(*args, **kwargs) [ 759.993754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 759.993754] nova-conductor[52800]: selections = self._select_destinations( [ 759.993754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 759.993754] nova-conductor[52800]: selections = self._schedule( [ 759.993754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 759.993754] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 759.993754] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 759.993754] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 759.993754] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 759.993754] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 759.993754] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-4e57a8f4-5cdf-46a1-b896-5e61790d0e8e tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 9aed678d-de26-42bd-93df-996e1b4019eb] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 761.169081] nova-conductor[52799]: Traceback (most recent call last): [ 761.169081] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 761.169081] nova-conductor[52799]: return func(*args, **kwargs) [ 761.169081] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 761.169081] nova-conductor[52799]: selections = self._select_destinations( [ 761.169081] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 761.169081] nova-conductor[52799]: selections = self._schedule( [ 761.169081] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 761.169081] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 761.169081] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 761.169081] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 761.169081] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.169081] nova-conductor[52799]: ERROR nova.conductor.manager [ 761.180153] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 761.180369] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 761.180544] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 761.241169] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] [instance: 1ec2719e-2473-477a-92d7-8fba75e97684] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 761.241969] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 761.242213] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 761.242377] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 761.249917] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 761.249917] nova-conductor[52799]: Traceback (most recent call last): [ 761.249917] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 761.249917] nova-conductor[52799]: return func(*args, **kwargs) [ 761.249917] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 761.249917] nova-conductor[52799]: selections = self._select_destinations( [ 761.249917] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 761.249917] nova-conductor[52799]: selections = self._schedule( [ 761.249917] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 761.249917] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 761.249917] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 761.249917] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 761.249917] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 761.249917] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 761.250503] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-dfd26c06-0139-4761-950c-d6d02e06649d tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] [instance: 1ec2719e-2473-477a-92d7-8fba75e97684] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.317372] nova-conductor[52800]: Traceback (most recent call last): [ 762.317372] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.317372] nova-conductor[52800]: return func(*args, **kwargs) [ 762.317372] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.317372] nova-conductor[52800]: selections = self._select_destinations( [ 762.317372] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.317372] nova-conductor[52800]: selections = self._schedule( [ 762.317372] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.317372] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 762.317372] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.317372] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 762.317372] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.317372] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.328834] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 762.328834] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 762.328834] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 762.460710] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: a5f4fbea-948f-4e98-8e5f-7348eb6c652b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 762.464366] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 762.464366] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 762.464366] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 762.469028] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 762.469028] nova-conductor[52800]: Traceback (most recent call last): [ 762.469028] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.469028] nova-conductor[52800]: return func(*args, **kwargs) [ 762.469028] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.469028] nova-conductor[52800]: selections = self._select_destinations( [ 762.469028] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.469028] nova-conductor[52800]: selections = self._schedule( [ 762.469028] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.469028] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 762.469028] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.469028] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 762.469028] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.469028] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.469028] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-3cd78985-4d5b-4a75-a66e-1e92a364fe06 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: a5f4fbea-948f-4e98-8e5f-7348eb6c652b] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.485253] nova-conductor[52799]: Traceback (most recent call last): [ 762.485253] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.485253] nova-conductor[52799]: return func(*args, **kwargs) [ 762.485253] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.485253] nova-conductor[52799]: selections = self._select_destinations( [ 762.485253] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.485253] nova-conductor[52799]: selections = self._schedule( [ 762.485253] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.485253] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 762.485253] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.485253] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 762.485253] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.485253] nova-conductor[52799]: ERROR nova.conductor.manager [ 762.497447] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 762.497447] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 762.497447] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 762.547050] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 2a5a318e-8a19-4375-ade9-e019cd04cae8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 762.547773] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 762.548201] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 762.548806] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 762.552840] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 762.552840] nova-conductor[52799]: Traceback (most recent call last): [ 762.552840] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.552840] nova-conductor[52799]: return func(*args, **kwargs) [ 762.552840] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.552840] nova-conductor[52799]: selections = self._select_destinations( [ 762.552840] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.552840] nova-conductor[52799]: selections = self._schedule( [ 762.552840] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.552840] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 762.552840] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.552840] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 762.552840] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.552840] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.553373] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-281992fb-2f1b-4d61-a17f-6209f4299e8d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 2a5a318e-8a19-4375-ade9-e019cd04cae8] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.706942] nova-conductor[52800]: Traceback (most recent call last): [ 762.706942] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.706942] nova-conductor[52800]: return func(*args, **kwargs) [ 762.706942] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.706942] nova-conductor[52800]: selections = self._select_destinations( [ 762.706942] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.706942] nova-conductor[52800]: selections = self._schedule( [ 762.706942] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.706942] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 762.706942] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.706942] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 762.706942] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.706942] nova-conductor[52800]: ERROR nova.conductor.manager [ 762.723737] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 762.723737] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 762.723737] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 762.800023] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] [instance: ab7961b6-8449-4b66-83d4-e251b3305a36] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 762.800023] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 762.800023] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 762.800023] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 762.802544] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 762.802544] nova-conductor[52800]: Traceback (most recent call last): [ 762.802544] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 762.802544] nova-conductor[52800]: return func(*args, **kwargs) [ 762.802544] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 762.802544] nova-conductor[52800]: selections = self._select_destinations( [ 762.802544] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 762.802544] nova-conductor[52800]: selections = self._schedule( [ 762.802544] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 762.802544] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 762.802544] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 762.802544] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 762.802544] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 762.802544] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 762.803074] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-73eb36a3-a5a6-483c-be1a-5d5186fff2c3 tempest-ServerShowV257Test-1850003832 tempest-ServerShowV257Test-1850003832-project-member] [instance: ab7961b6-8449-4b66-83d4-e251b3305a36] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 763.758298] nova-conductor[52799]: Traceback (most recent call last): [ 763.758298] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 763.758298] nova-conductor[52799]: return func(*args, **kwargs) [ 763.758298] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 763.758298] nova-conductor[52799]: selections = self._select_destinations( [ 763.758298] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 763.758298] nova-conductor[52799]: selections = self._schedule( [ 763.758298] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 763.758298] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 763.758298] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 763.758298] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 763.758298] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.758298] nova-conductor[52799]: ERROR nova.conductor.manager [ 763.767483] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.767731] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.767914] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 763.817961] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 64ef20cf-5e57-4c41-bca1-7f68ed15e0ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 763.820949] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 763.820949] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 763.820949] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 763.823114] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 763.823114] nova-conductor[52799]: Traceback (most recent call last): [ 763.823114] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 763.823114] nova-conductor[52799]: return func(*args, **kwargs) [ 763.823114] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 763.823114] nova-conductor[52799]: selections = self._select_destinations( [ 763.823114] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 763.823114] nova-conductor[52799]: selections = self._schedule( [ 763.823114] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 763.823114] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 763.823114] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 763.823114] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 763.823114] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 763.823114] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 763.823635] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5350268e-1579-461b-8475-8ac7b486df97 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 64ef20cf-5e57-4c41-bca1-7f68ed15e0ed] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.171025] nova-conductor[52800]: Traceback (most recent call last): [ 764.171025] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.171025] nova-conductor[52800]: return func(*args, **kwargs) [ 764.171025] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.171025] nova-conductor[52800]: selections = self._select_destinations( [ 764.171025] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.171025] nova-conductor[52800]: selections = self._schedule( [ 764.171025] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.171025] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 764.171025] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.171025] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 764.171025] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.171025] nova-conductor[52800]: ERROR nova.conductor.manager [ 764.177155] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 764.177433] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 764.177549] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 764.558348] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] [instance: ceee2fd6-7005-45ec-93e1-385543fbe412] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 764.558348] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 764.558348] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 764.558348] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 764.559519] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 764.559519] nova-conductor[52800]: Traceback (most recent call last): [ 764.559519] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 764.559519] nova-conductor[52800]: return func(*args, **kwargs) [ 764.559519] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 764.559519] nova-conductor[52800]: selections = self._select_destinations( [ 764.559519] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 764.559519] nova-conductor[52800]: selections = self._schedule( [ 764.559519] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 764.559519] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 764.559519] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 764.559519] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 764.559519] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 764.559519] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 764.560094] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-8c33cade-6f97-4f98-a179-6e420019afe0 tempest-ServerAddressesTestJSON-1896814982 tempest-ServerAddressesTestJSON-1896814982-project-member] [instance: ceee2fd6-7005-45ec-93e1-385543fbe412] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.148198] nova-conductor[52799]: Traceback (most recent call last): [ 766.148198] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 766.148198] nova-conductor[52799]: return func(*args, **kwargs) [ 766.148198] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 766.148198] nova-conductor[52799]: selections = self._select_destinations( [ 766.148198] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 766.148198] nova-conductor[52799]: selections = self._schedule( [ 766.148198] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 766.148198] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 766.148198] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 766.148198] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 766.148198] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.148198] nova-conductor[52799]: ERROR nova.conductor.manager [ 766.156930] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 766.156930] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 766.156930] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 766.220037] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 70bfc91c-9519-4b81-907d-d6e0da0b7e88] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 766.220774] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 766.220999] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 766.221182] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 766.224441] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 766.224441] nova-conductor[52799]: Traceback (most recent call last): [ 766.224441] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 766.224441] nova-conductor[52799]: return func(*args, **kwargs) [ 766.224441] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 766.224441] nova-conductor[52799]: selections = self._select_destinations( [ 766.224441] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 766.224441] nova-conductor[52799]: selections = self._schedule( [ 766.224441] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 766.224441] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 766.224441] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 766.224441] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 766.224441] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 766.224441] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.225676] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-5196af9e-1820-4180-85e0-ce9a205da864 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 70bfc91c-9519-4b81-907d-d6e0da0b7e88] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.822707] nova-conductor[52800]: Traceback (most recent call last): [ 766.822707] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 766.822707] nova-conductor[52800]: return func(*args, **kwargs) [ 766.822707] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 766.822707] nova-conductor[52800]: selections = self._select_destinations( [ 766.822707] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 766.822707] nova-conductor[52800]: selections = self._schedule( [ 766.822707] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 766.822707] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 766.822707] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 766.822707] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 766.822707] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.822707] nova-conductor[52800]: ERROR nova.conductor.manager [ 766.833108] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 766.834988] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 766.834988] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 766.876413] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] [instance: fe60d841-5406-4501-beb6-0c9dadd78edc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 766.877121] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 766.877587] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 766.877587] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 766.881595] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 766.881595] nova-conductor[52800]: Traceback (most recent call last): [ 766.881595] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 766.881595] nova-conductor[52800]: return func(*args, **kwargs) [ 766.881595] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 766.881595] nova-conductor[52800]: selections = self._select_destinations( [ 766.881595] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 766.881595] nova-conductor[52800]: selections = self._schedule( [ 766.881595] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 766.881595] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 766.881595] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 766.881595] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 766.881595] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 766.881595] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 766.881595] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-a38fa1a7-3f6a-495e-9b81-945afebf86da tempest-AttachVolumeNegativeTest-541211137 tempest-AttachVolumeNegativeTest-541211137-project-member] [instance: fe60d841-5406-4501-beb6-0c9dadd78edc] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.710692] nova-conductor[52799]: Traceback (most recent call last): [ 767.710692] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.710692] nova-conductor[52799]: return func(*args, **kwargs) [ 767.710692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.710692] nova-conductor[52799]: selections = self._select_destinations( [ 767.710692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.710692] nova-conductor[52799]: selections = self._schedule( [ 767.710692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.710692] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 767.710692] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.710692] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 767.710692] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.710692] nova-conductor[52799]: ERROR nova.conductor.manager [ 767.719938] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 767.720339] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 767.720421] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 767.789243] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 4c956deb-b893-4e03-a289-6b138e0310e6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 767.790030] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 767.790254] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 767.790425] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 767.796103] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 767.796103] nova-conductor[52799]: Traceback (most recent call last): [ 767.796103] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.796103] nova-conductor[52799]: return func(*args, **kwargs) [ 767.796103] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.796103] nova-conductor[52799]: selections = self._select_destinations( [ 767.796103] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.796103] nova-conductor[52799]: selections = self._schedule( [ 767.796103] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.796103] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 767.796103] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.796103] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 767.796103] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.796103] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.796103] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-fc7b48cc-d199-4d94-9dd1-fb684a84b733 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 4c956deb-b893-4e03-a289-6b138e0310e6] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.800656] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Acquiring lock "b5105805-148c-4545-8985-397f7b32e247" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 767.800871] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 767.801050] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Lock "b5105805-148c-4545-8985-397f7b32e247" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.906346] nova-conductor[52800]: Traceback (most recent call last): [ 767.906346] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.906346] nova-conductor[52800]: return func(*args, **kwargs) [ 767.906346] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.906346] nova-conductor[52800]: selections = self._select_destinations( [ 767.906346] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.906346] nova-conductor[52800]: selections = self._schedule( [ 767.906346] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.906346] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 767.906346] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.906346] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 767.906346] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.906346] nova-conductor[52800]: ERROR nova.conductor.manager [ 767.914549] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 767.914767] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 767.914935] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 767.984176] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] [instance: d53a7c81-c6b6-4a2a-a2f6-e9f659cb9f77] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 767.986024] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 767.986024] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 767.986024] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 767.993472] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 767.993472] nova-conductor[52800]: Traceback (most recent call last): [ 767.993472] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 767.993472] nova-conductor[52800]: return func(*args, **kwargs) [ 767.993472] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 767.993472] nova-conductor[52800]: selections = self._select_destinations( [ 767.993472] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 767.993472] nova-conductor[52800]: selections = self._schedule( [ 767.993472] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 767.993472] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 767.993472] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 767.993472] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 767.993472] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 767.993472] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 767.994379] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-3925fb09-acf4-4b01-8efa-0b7899f4ec9c tempest-ServerGroupTestJSON-1551242793 tempest-ServerGroupTestJSON-1551242793-project-member] [instance: d53a7c81-c6b6-4a2a-a2f6-e9f659cb9f77] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 769.133768] nova-conductor[52799]: Traceback (most recent call last): [ 769.133768] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 769.133768] nova-conductor[52799]: return func(*args, **kwargs) [ 769.133768] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 769.133768] nova-conductor[52799]: selections = self._select_destinations( [ 769.133768] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 769.133768] nova-conductor[52799]: selections = self._schedule( [ 769.133768] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 769.133768] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 769.133768] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 769.133768] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 769.133768] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.133768] nova-conductor[52799]: ERROR nova.conductor.manager [ 769.141077] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 769.141492] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 769.141492] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 769.185841] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 11e66c18-f38b-472e-bd0d-9889280f59bf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 769.186578] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 769.186778] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 769.186942] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 769.194665] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 769.194665] nova-conductor[52799]: Traceback (most recent call last): [ 769.194665] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 769.194665] nova-conductor[52799]: return func(*args, **kwargs) [ 769.194665] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 769.194665] nova-conductor[52799]: selections = self._select_destinations( [ 769.194665] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 769.194665] nova-conductor[52799]: selections = self._schedule( [ 769.194665] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 769.194665] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 769.194665] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 769.194665] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 769.194665] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 769.194665] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 769.195372] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-20927d0b-d36f-48b8-bc09-6941cea3b578 tempest-DeleteServersTestJSON-1725733590 tempest-DeleteServersTestJSON-1725733590-project-member] [instance: 11e66c18-f38b-472e-bd0d-9889280f59bf] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.138344] nova-conductor[52800]: Traceback (most recent call last): [ 770.138344] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.138344] nova-conductor[52800]: return func(*args, **kwargs) [ 770.138344] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.138344] nova-conductor[52800]: selections = self._select_destinations( [ 770.138344] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.138344] nova-conductor[52800]: selections = self._schedule( [ 770.138344] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.138344] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 770.138344] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.138344] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 770.138344] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.138344] nova-conductor[52800]: ERROR nova.conductor.manager [ 770.149314] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.149716] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.149934] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.209189] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 73e31978-4d20-421d-8fe6-e33e966f764f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 770.210024] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 770.210265] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 770.210395] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 770.214038] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 770.214038] nova-conductor[52800]: Traceback (most recent call last): [ 770.214038] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 770.214038] nova-conductor[52800]: return func(*args, **kwargs) [ 770.214038] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 770.214038] nova-conductor[52800]: selections = self._select_destinations( [ 770.214038] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 770.214038] nova-conductor[52800]: selections = self._schedule( [ 770.214038] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 770.214038] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 770.214038] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 770.214038] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 770.214038] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 770.214038] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 770.214974] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-9bbdb590-90eb-4117-b1c0-975f69847a98 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: 73e31978-4d20-421d-8fe6-e33e966f764f] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 771.543112] nova-conductor[52799]: Traceback (most recent call last): [ 771.543112] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 771.543112] nova-conductor[52799]: return func(*args, **kwargs) [ 771.543112] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 771.543112] nova-conductor[52799]: selections = self._select_destinations( [ 771.543112] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 771.543112] nova-conductor[52799]: selections = self._schedule( [ 771.543112] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 771.543112] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 771.543112] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 771.543112] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 771.543112] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.543112] nova-conductor[52799]: ERROR nova.conductor.manager [ 771.551260] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 771.552312] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 771.552691] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 771.604956] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] [instance: 9094c607-f110-4533-9e24-f9dc25f557d1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 771.605957] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 771.605957] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 771.606141] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 771.609651] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 771.609651] nova-conductor[52799]: Traceback (most recent call last): [ 771.609651] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 771.609651] nova-conductor[52799]: return func(*args, **kwargs) [ 771.609651] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 771.609651] nova-conductor[52799]: selections = self._select_destinations( [ 771.609651] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 771.609651] nova-conductor[52799]: selections = self._schedule( [ 771.609651] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 771.609651] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 771.609651] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 771.609651] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 771.609651] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 771.609651] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 771.610465] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-4914aedf-0759-4d4d-add3-0c3af8ea2660 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] [instance: 9094c607-f110-4533-9e24-f9dc25f557d1] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.680059] nova-conductor[52800]: Traceback (most recent call last): [ 772.680059] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.680059] nova-conductor[52800]: return func(*args, **kwargs) [ 772.680059] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.680059] nova-conductor[52800]: selections = self._select_destinations( [ 772.680059] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.680059] nova-conductor[52800]: selections = self._schedule( [ 772.680059] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.680059] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 772.680059] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.680059] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 772.680059] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.680059] nova-conductor[52800]: ERROR nova.conductor.manager [ 772.696255] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 772.696473] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 772.696635] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 772.739910] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: ab4c5ea3-d41b-4080-bba7-b092e8123c17] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 772.740645] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 772.740869] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 772.741077] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 772.744349] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 772.744349] nova-conductor[52800]: Traceback (most recent call last): [ 772.744349] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.744349] nova-conductor[52800]: return func(*args, **kwargs) [ 772.744349] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.744349] nova-conductor[52800]: selections = self._select_destinations( [ 772.744349] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.744349] nova-conductor[52800]: selections = self._schedule( [ 772.744349] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.744349] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 772.744349] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.744349] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 772.744349] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.744349] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.744987] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-99e8ba86-514b-422b-9a56-17b4b42d559d tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: ab4c5ea3-d41b-4080-bba7-b092e8123c17] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.893970] nova-conductor[52799]: Traceback (most recent call last): [ 772.893970] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.893970] nova-conductor[52799]: return func(*args, **kwargs) [ 772.893970] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.893970] nova-conductor[52799]: selections = self._select_destinations( [ 772.893970] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.893970] nova-conductor[52799]: selections = self._schedule( [ 772.893970] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.893970] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 772.893970] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.893970] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 772.893970] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.893970] nova-conductor[52799]: ERROR nova.conductor.manager [ 772.901964] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 772.902370] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 772.902370] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 772.990022] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] [instance: 4c0362e0-8c9c-4b06-8bae-0660557858b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 772.990022] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 772.990022] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 772.990022] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 772.993947] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 772.993947] nova-conductor[52799]: Traceback (most recent call last): [ 772.993947] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 772.993947] nova-conductor[52799]: return func(*args, **kwargs) [ 772.993947] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 772.993947] nova-conductor[52799]: selections = self._select_destinations( [ 772.993947] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 772.993947] nova-conductor[52799]: selections = self._schedule( [ 772.993947] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 772.993947] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 772.993947] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 772.993947] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 772.993947] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 772.993947] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 772.997379] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-6518154c-fa6e-4279-809f-f2365f473454 tempest-ServersTestManualDisk-1383397774 tempest-ServersTestManualDisk-1383397774-project-member] [instance: 4c0362e0-8c9c-4b06-8bae-0660557858b2] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 775.170875] nova-conductor[52800]: Traceback (most recent call last): [ 775.170875] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 775.170875] nova-conductor[52800]: return func(*args, **kwargs) [ 775.170875] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 775.170875] nova-conductor[52800]: selections = self._select_destinations( [ 775.170875] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 775.170875] nova-conductor[52800]: selections = self._schedule( [ 775.170875] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 775.170875] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 775.170875] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 775.170875] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 775.170875] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.170875] nova-conductor[52800]: ERROR nova.conductor.manager [ 775.186572] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 775.186917] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 775.187464] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 775.247678] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: e0bb4fa8-2bc1-45b6-a591-e2436dbe794a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 775.248522] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 775.248790] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 775.249008] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 775.252073] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 775.252073] nova-conductor[52800]: Traceback (most recent call last): [ 775.252073] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 775.252073] nova-conductor[52800]: return func(*args, **kwargs) [ 775.252073] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 775.252073] nova-conductor[52800]: selections = self._select_destinations( [ 775.252073] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 775.252073] nova-conductor[52800]: selections = self._schedule( [ 775.252073] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 775.252073] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 775.252073] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 775.252073] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 775.252073] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 775.252073] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 775.253044] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-01e43b73-69ac-4666-8f01-099eb61916c5 tempest-ServersTestJSON-314879797 tempest-ServersTestJSON-314879797-project-member] [instance: e0bb4fa8-2bc1-45b6-a591-e2436dbe794a] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 775.955722] nova-conductor[52799]: Traceback (most recent call last): [ 775.955722] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 775.955722] nova-conductor[52799]: return func(*args, **kwargs) [ 775.955722] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 775.955722] nova-conductor[52799]: selections = self._select_destinations( [ 775.955722] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 775.955722] nova-conductor[52799]: selections = self._schedule( [ 775.955722] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 775.955722] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 775.955722] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 775.955722] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 775.955722] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.955722] nova-conductor[52799]: ERROR nova.conductor.manager [ 775.969119] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 775.969339] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 775.969505] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.013946] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] [instance: 237850de-f7f7-49cf-97dd-5dfac741a3a1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 776.014810] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 776.015076] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 776.015076] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.019088] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 776.019088] nova-conductor[52799]: Traceback (most recent call last): [ 776.019088] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.019088] nova-conductor[52799]: return func(*args, **kwargs) [ 776.019088] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.019088] nova-conductor[52799]: selections = self._select_destinations( [ 776.019088] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.019088] nova-conductor[52799]: selections = self._schedule( [ 776.019088] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.019088] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 776.019088] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.019088] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 776.019088] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.019088] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.019613] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-816db4c7-c060-4ede-8885-4cd878a9d8c7 tempest-AttachVolumeShelveTestJSON-473055563 tempest-AttachVolumeShelveTestJSON-473055563-project-member] [instance: 237850de-f7f7-49cf-97dd-5dfac741a3a1] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.586048] nova-conductor[52800]: Traceback (most recent call last): [ 776.586048] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.586048] nova-conductor[52800]: return func(*args, **kwargs) [ 776.586048] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.586048] nova-conductor[52800]: selections = self._select_destinations( [ 776.586048] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.586048] nova-conductor[52800]: selections = self._schedule( [ 776.586048] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.586048] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 776.586048] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.586048] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 776.586048] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.586048] nova-conductor[52800]: ERROR nova.conductor.manager [ 776.593769] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 776.594173] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 776.594665] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.633603] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] [instance: 11894f7f-d67d-4356-80ad-bb7c16464e12] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 776.636329] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 776.636329] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 776.636329] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 776.638077] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 776.638077] nova-conductor[52800]: Traceback (most recent call last): [ 776.638077] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 776.638077] nova-conductor[52800]: return func(*args, **kwargs) [ 776.638077] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 776.638077] nova-conductor[52800]: selections = self._select_destinations( [ 776.638077] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 776.638077] nova-conductor[52800]: selections = self._schedule( [ 776.638077] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 776.638077] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 776.638077] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 776.638077] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 776.638077] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 776.638077] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 776.639071] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-f8f32154-6518-492b-ac13-eb50db110a79 tempest-ServerTagsTestJSON-1052921131 tempest-ServerTagsTestJSON-1052921131-project-member] [instance: 11894f7f-d67d-4356-80ad-bb7c16464e12] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 777.819325] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] [instance: 68545276-63f2-4baf-8110-d3cc71686682] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 68545276-63f2-4baf-8110-d3cc71686682 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 777.819955] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 777.820197] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 68545276-63f2-4baf-8110-d3cc71686682.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 68545276-63f2-4baf-8110-d3cc71686682. [ 777.820420] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-95589ff0-2ae0-4df8-9925-f5d63a2665fe tempest-ServerDiagnosticsV248Test-1137552807 tempest-ServerDiagnosticsV248Test-1137552807-project-member] [instance: 68545276-63f2-4baf-8110-d3cc71686682] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 68545276-63f2-4baf-8110-d3cc71686682. [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 783.520806] nova-conductor[52799]: Traceback (most recent call last): [ 783.520806] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 783.520806] nova-conductor[52799]: return func(*args, **kwargs) [ 783.520806] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 783.520806] nova-conductor[52799]: selections = self._select_destinations( [ 783.520806] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 783.520806] nova-conductor[52799]: selections = self._schedule( [ 783.520806] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 783.520806] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 783.520806] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 783.520806] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 783.520806] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager result = self.transport._send( [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager raise result [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager Traceback (most recent call last): [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._select_destinations( [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager selections = self._schedule( [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.520806] nova-conductor[52799]: ERROR nova.conductor.manager [ 783.527722] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 783.527971] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 783.528158] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 783.579142] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] [instance: 7ce56d80-2e8f-4648-996a-045c20662dfd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='1618eb55-f00d-42a5-b978-e81e57855fb4',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=52799) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 783.579872] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 783.580115] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 783.580286] nova-conductor[52799]: DEBUG oslo_concurrency.lockutils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52799) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 783.582918] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 783.582918] nova-conductor[52799]: Traceback (most recent call last): [ 783.582918] nova-conductor[52799]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 783.582918] nova-conductor[52799]: return func(*args, **kwargs) [ 783.582918] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 783.582918] nova-conductor[52799]: selections = self._select_destinations( [ 783.582918] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 783.582918] nova-conductor[52799]: selections = self._schedule( [ 783.582918] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 783.582918] nova-conductor[52799]: self._ensure_sufficient_hosts( [ 783.582918] nova-conductor[52799]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 783.582918] nova-conductor[52799]: raise exception.NoValidHost(reason=reason) [ 783.582918] nova-conductor[52799]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 783.582918] nova-conductor[52799]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 783.583440] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-7da7e836-3756-4583-a8b7-85eefbba1138 tempest-InstanceActionsNegativeTestJSON-702990142 tempest-InstanceActionsNegativeTestJSON-702990142-project-member] [instance: 7ce56d80-2e8f-4648-996a-045c20662dfd] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Failed to schedule instances: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 784.268273] nova-conductor[52800]: Traceback (most recent call last): [ 784.268273] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 784.268273] nova-conductor[52800]: return func(*args, **kwargs) [ 784.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 784.268273] nova-conductor[52800]: selections = self._select_destinations( [ 784.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 784.268273] nova-conductor[52800]: selections = self._schedule( [ 784.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 784.268273] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 784.268273] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 784.268273] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 784.268273] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 1601, in schedule_and_build_instances [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self._schedule_instances(context, request_specs[0], [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/conductor/manager.py", line 940, in _schedule_instances [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager host_lists = self.query_client.select_destinations( [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/client/query.py", line 41, in select_destinations [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager return self.scheduler_rpcapi.select_destinations(context, spec_obj, [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/rpcapi.py", line 160, in select_destinations [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager return cctxt.call(ctxt, 'select_destinations', **msg_args) [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/client.py", line 190, in call [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager result = self.transport._send( [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/transport.py", line 123, in _send [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager return self._driver.send(target, ctxt, message, [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 689, in send [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager return self._send(target, ctxt, message, wait_for_reply, timeout, [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 681, in _send [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager raise result [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager Traceback (most recent call last): [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager return func(*args, **kwargs) [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._select_destinations( [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager selections = self._schedule( [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager self._ensure_sufficient_hosts( [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager raise exception.NoValidHost(reason=reason) [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.268273] nova-conductor[52800]: ERROR nova.conductor.manager [ 784.275222] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 784.275447] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 784.275613] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 784.316014] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] [instance: 02b4c855-59ee-4e04-b583-73ee090610ba] block_device_mapping [BlockDeviceMapping(attachment_id=c14b676b-5ee3-42d8-9a6f-ba7b62589d13,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='3c5c8026-f905-4160-9634-b787a20c44f0',volume_size=1,volume_type=None)] {{(pid=52800) _create_block_device_mapping /opt/stack/nova/nova/conductor/manager.py:1453}} [ 784.316702] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} [ 784.316910] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} [ 784.317092] nova-conductor[52800]: DEBUG oslo_concurrency.lockutils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=52800) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} [ 784.320451] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] Failed to compute_task_build_instances: No valid host was found. There are not enough hosts available. [ 784.320451] nova-conductor[52800]: Traceback (most recent call last): [ 784.320451] nova-conductor[52800]: File "/usr/local/lib/python3.10/dist-packages/oslo_messaging/rpc/server.py", line 244, in inner [ 784.320451] nova-conductor[52800]: return func(*args, **kwargs) [ 784.320451] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 224, in select_destinations [ 784.320451] nova-conductor[52800]: selections = self._select_destinations( [ 784.320451] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 251, in _select_destinations [ 784.320451] nova-conductor[52800]: selections = self._schedule( [ 784.320451] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 452, in _schedule [ 784.320451] nova-conductor[52800]: self._ensure_sufficient_hosts( [ 784.320451] nova-conductor[52800]: File "/opt/stack/nova/nova/scheduler/manager.py", line 499, in _ensure_sufficient_hosts [ 784.320451] nova-conductor[52800]: raise exception.NoValidHost(reason=reason) [ 784.320451] nova-conductor[52800]: nova.exception.NoValidHost: No valid host was found. There are not enough hosts available. [ 784.320451] nova-conductor[52800]: : nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 784.321293] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-673da9f5-bb6f-4835-a532-6b46027168e3 tempest-ServerActionsV293TestJSON-2138665278 tempest-ServerActionsV293TestJSON-2138665278-project-member] [instance: 02b4c855-59ee-4e04-b583-73ee090610ba] Setting instance to ERROR state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. There are not enough hosts available. [ 824.997513] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: b1361aa5-9bbd-4891-b74f-a0afd90b0bd6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance b1361aa5-9bbd-4891-b74f-a0afd90b0bd6 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 824.998085] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 824.998316] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b1361aa5-9bbd-4891-b74f-a0afd90b0bd6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b1361aa5-9bbd-4891-b74f-a0afd90b0bd6. [ 824.998522] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-e12515a2-ed4a-472a-90a9-ba6e4491eab9 tempest-ServerShowV247Test-916977799 tempest-ServerShowV247Test-916977799-project-member] [instance: b1361aa5-9bbd-4891-b74f-a0afd90b0bd6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance b1361aa5-9bbd-4891-b74f-a0afd90b0bd6. [ 873.492233] nova-conductor[52800]: ERROR nova.scheduler.utils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] [instance: 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6 was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 873.492912] nova-conductor[52800]: DEBUG nova.conductor.manager [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Rescheduling: True {{(pid=52800) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 873.493048] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6. [ 873.493207] nova-conductor[52800]: WARNING nova.scheduler.utils [None req-d96c0ced-f062-463d-ae0d-4400b83d8169 tempest-ServerShowV254Test-893530536 tempest-ServerShowV254Test-893530536-project-member] [instance: 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 6c8194c3-68fd-4ffc-a0fa-f23c8935bee6. [ 920.175897] nova-conductor[52799]: ERROR nova.scheduler.utils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] [instance: 946c73f8-1ed8-4180-a9d7-0b2970c4367e] Error from last host: cpu-1 (node domain-c8.fc996f14-c53b-4953-92e3-bdfa48f5cc28): ['Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2607, in _build_and_run_instance\n self.driver.spawn(context, instance, image_meta,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 529, in spawn\n self._vmops.spawn(context, instance, image_meta, injected_files,\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 786, in spawn\n self._fetch_image_if_missing(context, vi)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 639, in _fetch_image_if_missing\n image_cache(vi, tmp_image_ds_loc)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 537, in _cache_sparse_image\n vm_util.copy_virtual_disk(\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/vm_util.py", line 1423, in copy_virtual_disk\n session._wait_for_task(vmdk_copy_task)\n', ' File "/opt/stack/nova/nova/virt/vmwareapi/session.py", line 157, in _wait_for_task\n return self.wait_for_task(task_ref)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 398, in wait_for_task\n return evt.wait()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/common/loopingcall.py", line 75, in _inner\n self.f(*self.args, **self.kw)\n', ' File "/usr/local/lib/python3.10/dist-packages/oslo_vmware/api.py", line 448, in _poll_task\n raise exceptions.translate_fault(task_info.error)\n', "oslo_vmware.exceptions.VimFaultException: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n", '\nDuring handling of the above exception, another exception occurred:\n\n', 'Traceback (most recent call last):\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2428, in _do_build_and_run_instance\n self._build_and_run_instance(context, instance, image,\n', ' File "/opt/stack/nova/nova/compute/manager.py", line 2703, in _build_and_run_instance\n raise exception.RescheduledException(\n', "nova.exception.RescheduledException: Build of instance 946c73f8-1ed8-4180-a9d7-0b2970c4367e was re-scheduled: A specified parameter was not correct: fileType\nFaults: ['InvalidArgument']\n"] [ 920.176500] nova-conductor[52799]: DEBUG nova.conductor.manager [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Rescheduling: True {{(pid=52799) build_instances /opt/stack/nova/nova/conductor/manager.py:693}} [ 920.176735] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] Failed to compute_task_build_instances: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 946c73f8-1ed8-4180-a9d7-0b2970c4367e.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 946c73f8-1ed8-4180-a9d7-0b2970c4367e. [ 920.176949] nova-conductor[52799]: WARNING nova.scheduler.utils [None req-59a315b4-ab5c-4345-bdbe-05be08a4aa67 tempest-ServersAaction247Test-439800747 tempest-ServersAaction247Test-439800747-project-member] [instance: 946c73f8-1ed8-4180-a9d7-0b2970c4367e] Setting instance to ERROR state.: nova.exception.MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 946c73f8-1ed8-4180-a9d7-0b2970c4367e.