[ 461.147425] env[61025]: INFO neutron.common.config [-] Logging enabled! [ 461.147936] env[61025]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 461.147936] env[61025]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61025) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 461.148147] env[61025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 461.148218] env[61025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 461.149031] env[61025]: WARNING oslo_policy.policy [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 461.149251] env[61025]: WARNING oslo_policy.policy [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 461.150388] env[61025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 461.169055] env[61025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Session token - acquiring [ 461.169813] env[61025]: DEBUG networking_nsxv3.common.locking [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61025) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 461.170012] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 461.170137] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 461.170518] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 461.180672] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 461.180917] env[61025]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 461.181063] env[61025]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61025) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 463.187427] env[61025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Session token - acquiring [ 463.187776] env[61025]: DEBUG networking_nsxv3.common.locking [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61025) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 463.187931] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 463.188084] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 463.188363] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 463.197466] env[61025]: DEBUG oslo_concurrency.lockutils [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 463.197691] env[61025]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 463.197830] env[61025]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61025) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 465.200758] env[61025]: CRITICAL neutron [None req-b7b006e4-e861-49ec-a7af-c49be5a48b73 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 465.200758] env[61025]: ERROR neutron Traceback (most recent call last): [ 465.200758] env[61025]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 465.200758] env[61025]: ERROR neutron sys.exit(main()) [ 465.200758] env[61025]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 465.200758] env[61025]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 465.200758] env[61025]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 465.200758] env[61025]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 465.200758] env[61025]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 465.200758] env[61025]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 465.200758] env[61025]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 465.200758] env[61025]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 465.200758] env[61025]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 465.200758] env[61025]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 465.200758] env[61025]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 465.200758] env[61025]: ERROR neutron response = self.get(path=path, params=params) [ 465.201390] env[61025]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 465.201390] env[61025]: ERROR neutron raise RuntimeError(msg, last_err) [ 465.201390] env[61025]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 465.201390] env[61025]: ERROR neutron [ 467.632431] env[61060]: INFO neutron.common.config [-] Logging enabled! [ 467.632781] env[61060]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 467.632920] env[61060]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61060) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 467.633142] env[61060]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 467.633232] env[61060]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 467.634060] env[61060]: WARNING oslo_policy.policy [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 467.634274] env[61060]: WARNING oslo_policy.policy [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 467.635467] env[61060]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 467.650492] env[61060]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Session token - acquiring [ 467.651183] env[61060]: DEBUG networking_nsxv3.common.locking [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61060) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 467.651378] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 467.651502] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 467.651851] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 467.660873] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 467.661267] env[61060]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 467.661412] env[61060]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61060) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 469.667355] env[61060]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Session token - acquiring [ 469.667655] env[61060]: DEBUG networking_nsxv3.common.locking [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61060) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 469.667850] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 469.667974] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 469.668269] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 469.676924] env[61060]: DEBUG oslo_concurrency.lockutils [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61060) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 469.677150] env[61060]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 469.677285] env[61060]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61060) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 471.680233] env[61060]: CRITICAL neutron [None req-fd1ffb19-ed0f-48f3-9902-1d25734cb963 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 471.680233] env[61060]: ERROR neutron Traceback (most recent call last): [ 471.680233] env[61060]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 471.680233] env[61060]: ERROR neutron sys.exit(main()) [ 471.680233] env[61060]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 471.680233] env[61060]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 471.680233] env[61060]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 471.680233] env[61060]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 471.680233] env[61060]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 471.680233] env[61060]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 471.680233] env[61060]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 471.680233] env[61060]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 471.680233] env[61060]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 471.680233] env[61060]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 471.680233] env[61060]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 471.680233] env[61060]: ERROR neutron response = self.get(path=path, params=params) [ 471.680956] env[61060]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 471.680956] env[61060]: ERROR neutron raise RuntimeError(msg, last_err) [ 471.680956] env[61060]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 471.680956] env[61060]: ERROR neutron [ 474.146092] env[61084]: INFO neutron.common.config [-] Logging enabled! [ 474.146528] env[61084]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 474.146568] env[61084]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61084) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 474.146814] env[61084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 474.146923] env[61084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 474.147739] env[61084]: WARNING oslo_policy.policy [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 474.147958] env[61084]: WARNING oslo_policy.policy [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 474.149198] env[61084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 474.164052] env[61084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Session token - acquiring [ 474.164798] env[61084]: DEBUG networking_nsxv3.common.locking [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61084) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 474.164998] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 474.165123] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 474.165474] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 474.174522] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 474.174757] env[61084]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 474.174903] env[61084]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61084) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 476.181118] env[61084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Session token - acquiring [ 476.181449] env[61084]: DEBUG networking_nsxv3.common.locking [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61084) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 476.181610] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 476.181733] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 476.181998] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 476.191060] env[61084]: DEBUG oslo_concurrency.lockutils [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 476.191287] env[61084]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 476.191428] env[61084]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61084) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 478.194454] env[61084]: CRITICAL neutron [None req-00b96096-1d3d-40d3-97ae-a3773d39ef87 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 478.194454] env[61084]: ERROR neutron Traceback (most recent call last): [ 478.194454] env[61084]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 478.194454] env[61084]: ERROR neutron sys.exit(main()) [ 478.194454] env[61084]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 478.194454] env[61084]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 478.194454] env[61084]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 478.194454] env[61084]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 478.194454] env[61084]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 478.194454] env[61084]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 478.194454] env[61084]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 478.194454] env[61084]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 478.194454] env[61084]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 478.194454] env[61084]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 478.194454] env[61084]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 478.194454] env[61084]: ERROR neutron response = self.get(path=path, params=params) [ 478.195219] env[61084]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 478.195219] env[61084]: ERROR neutron raise RuntimeError(msg, last_err) [ 478.195219] env[61084]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 478.195219] env[61084]: ERROR neutron [ 480.666703] env[61129]: INFO neutron.common.config [-] Logging enabled! [ 480.667155] env[61129]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 480.667155] env[61129]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61129) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 480.667367] env[61129]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 480.667462] env[61129]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 480.668316] env[61129]: WARNING oslo_policy.policy [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 480.668539] env[61129]: WARNING oslo_policy.policy [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 480.669680] env[61129]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 480.683592] env[61129]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Session token - acquiring [ 480.684351] env[61129]: DEBUG networking_nsxv3.common.locking [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61129) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 480.684547] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 480.684675] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 480.685029] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 480.693772] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 480.694015] env[61129]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 480.694162] env[61129]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61129) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 482.700477] env[61129]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Session token - acquiring [ 482.700852] env[61129]: DEBUG networking_nsxv3.common.locking [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61129) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 482.700999] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 482.701105] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 482.701393] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 482.710650] env[61129]: DEBUG oslo_concurrency.lockutils [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61129) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 482.710886] env[61129]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 482.711029] env[61129]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61129) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 484.714084] env[61129]: CRITICAL neutron [None req-3c4f3808-853e-4637-9798-9686ec5dec52 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 484.714084] env[61129]: ERROR neutron Traceback (most recent call last): [ 484.714084] env[61129]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 484.714084] env[61129]: ERROR neutron sys.exit(main()) [ 484.714084] env[61129]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 484.714084] env[61129]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 484.714084] env[61129]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 484.714084] env[61129]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 484.714084] env[61129]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 484.714084] env[61129]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 484.714084] env[61129]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 484.714084] env[61129]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 484.714084] env[61129]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 484.714084] env[61129]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 484.714084] env[61129]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 484.714084] env[61129]: ERROR neutron response = self.get(path=path, params=params) [ 484.714834] env[61129]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 484.714834] env[61129]: ERROR neutron raise RuntimeError(msg, last_err) [ 484.714834] env[61129]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 484.714834] env[61129]: ERROR neutron [ 487.115305] env[61136]: INFO neutron.common.config [-] Logging enabled! [ 487.115699] env[61136]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 487.115796] env[61136]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61136) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 487.116035] env[61136]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 487.116133] env[61136]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 487.116932] env[61136]: WARNING oslo_policy.policy [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 487.117144] env[61136]: WARNING oslo_policy.policy [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 487.118292] env[61136]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 487.132086] env[61136]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Session token - acquiring [ 487.132817] env[61136]: DEBUG networking_nsxv3.common.locking [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61136) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 487.133042] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 487.133173] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 487.133530] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 487.142839] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 487.143083] env[61136]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 487.143230] env[61136]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61136) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 489.149505] env[61136]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Session token - acquiring [ 489.150031] env[61136]: DEBUG networking_nsxv3.common.locking [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61136) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 489.150031] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 489.150249] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 489.150430] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 489.159945] env[61136]: DEBUG oslo_concurrency.lockutils [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61136) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 489.160190] env[61136]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 489.160336] env[61136]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61136) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 491.161666] env[61136]: CRITICAL neutron [None req-91a21898-0cbd-427c-9c64-82cf44484507 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 491.161666] env[61136]: ERROR neutron Traceback (most recent call last): [ 491.161666] env[61136]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 491.161666] env[61136]: ERROR neutron sys.exit(main()) [ 491.161666] env[61136]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 491.161666] env[61136]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 491.161666] env[61136]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 491.161666] env[61136]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 491.161666] env[61136]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 491.161666] env[61136]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 491.161666] env[61136]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 491.161666] env[61136]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 491.161666] env[61136]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 491.161666] env[61136]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 491.161666] env[61136]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 491.161666] env[61136]: ERROR neutron response = self.get(path=path, params=params) [ 491.162708] env[61136]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 491.162708] env[61136]: ERROR neutron raise RuntimeError(msg, last_err) [ 491.162708] env[61136]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 491.162708] env[61136]: ERROR neutron [ 493.653275] env[61144]: INFO neutron.common.config [-] Logging enabled! [ 493.653613] env[61144]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 493.653784] env[61144]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61144) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 493.654004] env[61144]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 493.654097] env[61144]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 493.654910] env[61144]: WARNING oslo_policy.policy [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 493.655133] env[61144]: WARNING oslo_policy.policy [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 493.656353] env[61144]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 493.670506] env[61144]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Session token - acquiring [ 493.671266] env[61144]: DEBUG networking_nsxv3.common.locking [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61144) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 493.671470] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 493.671598] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 493.671956] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 493.680920] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 493.681177] env[61144]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 493.681325] env[61144]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61144) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 495.687274] env[61144]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Session token - acquiring [ 495.687625] env[61144]: DEBUG networking_nsxv3.common.locking [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61144) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 495.687785] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 495.687861] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 495.688153] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 495.697180] env[61144]: DEBUG oslo_concurrency.lockutils [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61144) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 495.697397] env[61144]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 495.697536] env[61144]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61144) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 497.700448] env[61144]: CRITICAL neutron [None req-14980483-93ab-4e12-b09b-997d0ff5cf52 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 497.700448] env[61144]: ERROR neutron Traceback (most recent call last): [ 497.700448] env[61144]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 497.700448] env[61144]: ERROR neutron sys.exit(main()) [ 497.700448] env[61144]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 497.700448] env[61144]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 497.700448] env[61144]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 497.700448] env[61144]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 497.700448] env[61144]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 497.700448] env[61144]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 497.700448] env[61144]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 497.700448] env[61144]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 497.700448] env[61144]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 497.700448] env[61144]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 497.700448] env[61144]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 497.700448] env[61144]: ERROR neutron response = self.get(path=path, params=params) [ 497.701155] env[61144]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 497.701155] env[61144]: ERROR neutron raise RuntimeError(msg, last_err) [ 497.701155] env[61144]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 497.701155] env[61144]: ERROR neutron [ 500.418462] env[61417]: INFO neutron.common.config [-] Logging enabled! [ 500.418781] env[61417]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 500.419003] env[61417]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61417) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 500.419221] env[61417]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 500.419311] env[61417]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 500.420206] env[61417]: WARNING oslo_policy.policy [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 500.420414] env[61417]: WARNING oslo_policy.policy [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 500.421643] env[61417]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 500.438287] env[61417]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Session token - acquiring [ 500.439145] env[61417]: DEBUG networking_nsxv3.common.locking [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61417) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 500.439348] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 500.439471] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 500.439836] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 500.449519] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 500.449789] env[61417]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 500.449931] env[61417]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61417) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 502.457932] env[61417]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Session token - acquiring [ 502.458232] env[61417]: DEBUG networking_nsxv3.common.locking [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61417) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 502.458417] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 502.458539] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 502.458817] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 502.467616] env[61417]: DEBUG oslo_concurrency.lockutils [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61417) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 502.467853] env[61417]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 502.467989] env[61417]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61417) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 504.469692] env[61417]: CRITICAL neutron [None req-65f34b98-bb9d-4933-b327-6992dd596740 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 504.469692] env[61417]: ERROR neutron Traceback (most recent call last): [ 504.469692] env[61417]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 504.469692] env[61417]: ERROR neutron sys.exit(main()) [ 504.469692] env[61417]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 504.469692] env[61417]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 504.469692] env[61417]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 504.469692] env[61417]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 504.469692] env[61417]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 504.469692] env[61417]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 504.469692] env[61417]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 504.469692] env[61417]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 504.469692] env[61417]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 504.469692] env[61417]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 504.469692] env[61417]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 504.469692] env[61417]: ERROR neutron response = self.get(path=path, params=params) [ 504.470384] env[61417]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 504.470384] env[61417]: ERROR neutron raise RuntimeError(msg, last_err) [ 504.470384] env[61417]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 504.470384] env[61417]: ERROR neutron [ 507.068535] env[61456]: INFO neutron.common.config [-] Logging enabled! [ 507.068929] env[61456]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 507.069123] env[61456]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61456) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 507.069348] env[61456]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 507.069440] env[61456]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 507.070415] env[61456]: WARNING oslo_policy.policy [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 507.070648] env[61456]: WARNING oslo_policy.policy [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 507.073279] env[61456]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 507.090654] env[61456]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Session token - acquiring [ 507.095660] env[61456]: DEBUG networking_nsxv3.common.locking [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61456) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 507.095889] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 507.096084] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 507.097314] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 507.108984] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 507.109315] env[61456]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 507.109461] env[61456]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61456) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 509.116419] env[61456]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Session token - acquiring [ 509.116816] env[61456]: DEBUG networking_nsxv3.common.locking [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61456) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 509.117022] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 509.117219] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 509.117365] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 509.126868] env[61456]: DEBUG oslo_concurrency.lockutils [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61456) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 509.127241] env[61456]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 509.127319] env[61456]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61456) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 511.128972] env[61456]: CRITICAL neutron [None req-1fc15aca-236b-463b-a6a6-67a58beb5b7f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 511.128972] env[61456]: ERROR neutron Traceback (most recent call last): [ 511.128972] env[61456]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 511.128972] env[61456]: ERROR neutron sys.exit(main()) [ 511.128972] env[61456]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 511.128972] env[61456]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 511.128972] env[61456]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 511.128972] env[61456]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 511.128972] env[61456]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 511.128972] env[61456]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 511.128972] env[61456]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 511.128972] env[61456]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 511.128972] env[61456]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 511.128972] env[61456]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 511.128972] env[61456]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 511.128972] env[61456]: ERROR neutron response = self.get(path=path, params=params) [ 511.129653] env[61456]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 511.129653] env[61456]: ERROR neutron raise RuntimeError(msg, last_err) [ 511.129653] env[61456]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 511.129653] env[61456]: ERROR neutron [ 514.077944] env[61998]: INFO neutron.common.config [-] Logging enabled! [ 514.078233] env[61998]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 514.078523] env[61998]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61998) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 514.078774] env[61998]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 514.078877] env[61998]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 514.079788] env[61998]: WARNING oslo_policy.policy [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 514.080058] env[61998]: WARNING oslo_policy.policy [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 514.081617] env[61998]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 514.097567] env[61998]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Session token - acquiring [ 514.098473] env[61998]: DEBUG networking_nsxv3.common.locking [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61998) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 514.098682] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 514.098811] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 514.099196] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 514.109612] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 514.109812] env[61998]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 514.110001] env[61998]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61998) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 516.119910] env[61998]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Session token - acquiring [ 516.120274] env[61998]: DEBUG networking_nsxv3.common.locking [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61998) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 516.120453] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 516.120572] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 516.121194] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 516.130548] env[61998]: DEBUG oslo_concurrency.lockutils [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61998) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 516.130843] env[61998]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 516.130983] env[61998]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61998) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 518.132928] env[61998]: CRITICAL neutron [None req-8b6a55ea-d2ca-4295-b919-ea7afd113697 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 518.132928] env[61998]: ERROR neutron Traceback (most recent call last): [ 518.132928] env[61998]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 518.132928] env[61998]: ERROR neutron sys.exit(main()) [ 518.132928] env[61998]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 518.132928] env[61998]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 518.132928] env[61998]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 518.132928] env[61998]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 518.132928] env[61998]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 518.132928] env[61998]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 518.132928] env[61998]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 518.132928] env[61998]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 518.132928] env[61998]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 518.132928] env[61998]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 518.132928] env[61998]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 518.132928] env[61998]: ERROR neutron response = self.get(path=path, params=params) [ 518.133719] env[61998]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 518.133719] env[61998]: ERROR neutron raise RuntimeError(msg, last_err) [ 518.133719] env[61998]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 518.133719] env[61998]: ERROR neutron [ 521.074624] env[62533]: INFO neutron.common.config [-] Logging enabled! [ 521.074959] env[62533]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 521.075268] env[62533]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62533) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 521.075547] env[62533]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 521.075675] env[62533]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 521.076944] env[62533]: WARNING oslo_policy.policy [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 521.077053] env[62533]: WARNING oslo_policy.policy [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 521.078531] env[62533]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 521.096645] env[62533]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Session token - acquiring [ 521.099964] env[62533]: DEBUG networking_nsxv3.common.locking [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62533) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 521.099964] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 521.100179] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 521.100179] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 521.123041] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 521.123041] env[62533]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 521.123041] env[62533]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62533) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 523.125337] env[62533]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Session token - acquiring [ 523.125678] env[62533]: DEBUG networking_nsxv3.common.locking [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62533) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 523.125930] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 523.126051] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 523.126349] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 523.136105] env[62533]: DEBUG oslo_concurrency.lockutils [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62533) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 523.136371] env[62533]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 523.136516] env[62533]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62533) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 525.139478] env[62533]: CRITICAL neutron [None req-3b0837af-ecb6-48ef-8cba-c95b21d29eeb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 525.139478] env[62533]: ERROR neutron Traceback (most recent call last): [ 525.139478] env[62533]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 525.139478] env[62533]: ERROR neutron sys.exit(main()) [ 525.139478] env[62533]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 525.139478] env[62533]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 525.139478] env[62533]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 525.139478] env[62533]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 525.139478] env[62533]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 525.139478] env[62533]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 525.139478] env[62533]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 525.139478] env[62533]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 525.139478] env[62533]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 525.139478] env[62533]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 525.139478] env[62533]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 525.139478] env[62533]: ERROR neutron response = self.get(path=path, params=params) [ 525.140267] env[62533]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 525.140267] env[62533]: ERROR neutron raise RuntimeError(msg, last_err) [ 525.140267] env[62533]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 525.140267] env[62533]: ERROR neutron [ 527.861120] env[62873]: INFO neutron.common.config [-] Logging enabled! [ 527.861120] env[62873]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 527.861120] env[62873]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62873) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 527.861120] env[62873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 527.861120] env[62873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 527.861120] env[62873]: WARNING oslo_policy.policy [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 527.861120] env[62873]: WARNING oslo_policy.policy [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 527.862900] env[62873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 527.879981] env[62873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Session token - acquiring [ 527.881048] env[62873]: DEBUG networking_nsxv3.common.locking [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62873) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 527.881193] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 527.881193] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 527.881551] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 527.891406] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 527.891934] env[62873]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 527.892379] env[62873]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62873) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 529.898559] env[62873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Session token - acquiring [ 529.898906] env[62873]: DEBUG networking_nsxv3.common.locking [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62873) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 529.899131] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 529.899250] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 529.899519] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 529.911397] env[62873]: DEBUG oslo_concurrency.lockutils [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62873) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 529.911397] env[62873]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 529.911397] env[62873]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62873) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 531.912809] env[62873]: CRITICAL neutron [None req-046ca50f-4c28-446c-a345-4b7f872a570e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 531.912809] env[62873]: ERROR neutron Traceback (most recent call last): [ 531.912809] env[62873]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 531.912809] env[62873]: ERROR neutron sys.exit(main()) [ 531.912809] env[62873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 531.912809] env[62873]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 531.912809] env[62873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 531.912809] env[62873]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 531.912809] env[62873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 531.912809] env[62873]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 531.912809] env[62873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 531.912809] env[62873]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 531.912809] env[62873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 531.912809] env[62873]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 531.912809] env[62873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 531.912809] env[62873]: ERROR neutron response = self.get(path=path, params=params) [ 531.913540] env[62873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 531.913540] env[62873]: ERROR neutron raise RuntimeError(msg, last_err) [ 531.913540] env[62873]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 531.913540] env[62873]: ERROR neutron [ 534.670015] env[63445]: INFO neutron.common.config [-] Logging enabled! [ 534.670258] env[63445]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 534.670980] env[63445]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63445) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 534.671268] env[63445]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 534.671363] env[63445]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 534.672362] env[63445]: WARNING oslo_policy.policy [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 534.672578] env[63445]: WARNING oslo_policy.policy [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 534.673927] env[63445]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 534.697834] env[63445]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Session token - acquiring [ 534.698728] env[63445]: DEBUG networking_nsxv3.common.locking [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63445) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 534.699008] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 534.699087] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 534.699465] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 534.709644] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 534.709938] env[63445]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 534.710081] env[63445]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63445) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 536.718198] env[63445]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Session token - acquiring [ 536.718553] env[63445]: DEBUG networking_nsxv3.common.locking [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63445) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 536.718696] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 536.718814] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 536.719094] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 536.732626] env[63445]: DEBUG oslo_concurrency.lockutils [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63445) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 536.734046] env[63445]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 536.734046] env[63445]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63445) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 538.737437] env[63445]: CRITICAL neutron [None req-3785ff7f-bf6a-4678-86db-8f6828fd9e06 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 538.737437] env[63445]: ERROR neutron Traceback (most recent call last): [ 538.737437] env[63445]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 538.737437] env[63445]: ERROR neutron sys.exit(main()) [ 538.737437] env[63445]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 538.737437] env[63445]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 538.737437] env[63445]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 538.737437] env[63445]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 538.737437] env[63445]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 538.737437] env[63445]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 538.737437] env[63445]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 538.737437] env[63445]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 538.737437] env[63445]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 538.737437] env[63445]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 538.737437] env[63445]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 538.737437] env[63445]: ERROR neutron response = self.get(path=path, params=params) [ 538.738046] env[63445]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 538.738046] env[63445]: ERROR neutron raise RuntimeError(msg, last_err) [ 538.738046] env[63445]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 538.738046] env[63445]: ERROR neutron [ 541.809673] env[64013]: INFO neutron.common.config [-] Logging enabled! [ 541.810036] env[64013]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 541.810236] env[64013]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64013) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 541.810486] env[64013]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 541.810583] env[64013]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 541.811523] env[64013]: WARNING oslo_policy.policy [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 541.811764] env[64013]: WARNING oslo_policy.policy [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 541.813072] env[64013]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 541.828951] env[64013]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Session token - acquiring [ 541.829834] env[64013]: DEBUG networking_nsxv3.common.locking [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64013) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 541.830057] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 541.830231] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 541.830659] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 541.841092] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 541.841363] env[64013]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 541.841510] env[64013]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64013) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 543.847752] env[64013]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Session token - acquiring [ 543.848207] env[64013]: DEBUG networking_nsxv3.common.locking [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64013) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 543.848414] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 543.848551] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 543.848852] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 543.860156] env[64013]: DEBUG oslo_concurrency.lockutils [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64013) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 543.860475] env[64013]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 543.860624] env[64013]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64013) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 545.863513] env[64013]: CRITICAL neutron [None req-8711f99f-d951-46e1-89c5-d0f97763ffca None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 545.863513] env[64013]: ERROR neutron Traceback (most recent call last): [ 545.863513] env[64013]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 545.863513] env[64013]: ERROR neutron sys.exit(main()) [ 545.863513] env[64013]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 545.863513] env[64013]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 545.863513] env[64013]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 545.863513] env[64013]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 545.863513] env[64013]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 545.863513] env[64013]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 545.863513] env[64013]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 545.863513] env[64013]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 545.863513] env[64013]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 545.863513] env[64013]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 545.863513] env[64013]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 545.863513] env[64013]: ERROR neutron response = self.get(path=path, params=params) [ 545.864206] env[64013]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 545.864206] env[64013]: ERROR neutron raise RuntimeError(msg, last_err) [ 545.864206] env[64013]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 545.864206] env[64013]: ERROR neutron [ 548.623165] env[64537]: INFO neutron.common.config [-] Logging enabled! [ 548.623402] env[64537]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 548.623719] env[64537]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64537) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 548.623946] env[64537]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 548.624052] env[64537]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 548.626737] env[64537]: WARNING oslo_policy.policy [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 548.628607] env[64537]: WARNING oslo_policy.policy [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 548.629924] env[64537]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 548.645844] env[64537]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Session token - acquiring [ 548.645844] env[64537]: DEBUG networking_nsxv3.common.locking [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64537) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 548.645844] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 548.646002] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 548.646383] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 548.655892] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 548.656181] env[64537]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 548.656324] env[64537]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64537) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 550.662593] env[64537]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Session token - acquiring [ 550.662910] env[64537]: DEBUG networking_nsxv3.common.locking [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64537) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 550.663109] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 550.663236] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 550.663510] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 550.674988] env[64537]: DEBUG oslo_concurrency.lockutils [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64537) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 550.675251] env[64537]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 550.675667] env[64537]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64537) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 552.678200] env[64537]: CRITICAL neutron [None req-445c6515-a602-44cc-92d8-103c79f62f80 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 552.678200] env[64537]: ERROR neutron Traceback (most recent call last): [ 552.678200] env[64537]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 552.678200] env[64537]: ERROR neutron sys.exit(main()) [ 552.678200] env[64537]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 552.678200] env[64537]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 552.678200] env[64537]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 552.678200] env[64537]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 552.678200] env[64537]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 552.678200] env[64537]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 552.678200] env[64537]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 552.678200] env[64537]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 552.678200] env[64537]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 552.678200] env[64537]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 552.678200] env[64537]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 552.678200] env[64537]: ERROR neutron response = self.get(path=path, params=params) [ 552.678972] env[64537]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 552.678972] env[64537]: ERROR neutron raise RuntimeError(msg, last_err) [ 552.678972] env[64537]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 552.678972] env[64537]: ERROR neutron [ 555.696401] env[64871]: INFO neutron.common.config [-] Logging enabled! [ 555.696401] env[64871]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 555.696853] env[64871]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64871) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 555.697104] env[64871]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 555.697221] env[64871]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 555.698112] env[64871]: WARNING oslo_policy.policy [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 555.698356] env[64871]: WARNING oslo_policy.policy [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 555.700169] env[64871]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 555.714671] env[64871]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Session token - acquiring [ 555.715576] env[64871]: DEBUG networking_nsxv3.common.locking [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64871) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 555.716396] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 555.716396] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 555.716570] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 555.726703] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 555.727057] env[64871]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 555.727186] env[64871]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64871) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 557.733073] env[64871]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Session token - acquiring [ 557.733406] env[64871]: DEBUG networking_nsxv3.common.locking [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64871) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 557.733618] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 557.733758] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 557.734024] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 557.743648] env[64871]: DEBUG oslo_concurrency.lockutils [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64871) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 557.743901] env[64871]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 557.744054] env[64871]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64871) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 559.751109] env[64871]: CRITICAL neutron [None req-2053d8bc-8576-4958-ad5b-1ee82bb1e030 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 559.751109] env[64871]: ERROR neutron Traceback (most recent call last): [ 559.751109] env[64871]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 559.751109] env[64871]: ERROR neutron sys.exit(main()) [ 559.751109] env[64871]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 559.751109] env[64871]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 559.751109] env[64871]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 559.751109] env[64871]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 559.751109] env[64871]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 559.751109] env[64871]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 559.751109] env[64871]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 559.751109] env[64871]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 559.751109] env[64871]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 559.751109] env[64871]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 559.751109] env[64871]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 559.751109] env[64871]: ERROR neutron response = self.get(path=path, params=params) [ 559.752072] env[64871]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 559.752072] env[64871]: ERROR neutron raise RuntimeError(msg, last_err) [ 559.752072] env[64871]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 559.752072] env[64871]: ERROR neutron [ 562.624837] env[65184]: INFO neutron.common.config [-] Logging enabled! [ 562.625074] env[65184]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 562.625384] env[65184]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65184) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 562.625632] env[65184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 562.625725] env[65184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 562.627410] env[65184]: WARNING oslo_policy.policy [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 562.627632] env[65184]: WARNING oslo_policy.policy [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 562.629037] env[65184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 562.645127] env[65184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Session token - acquiring [ 562.646003] env[65184]: DEBUG networking_nsxv3.common.locking [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65184) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 562.646251] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 562.654238] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 562.654238] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 562.660048] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 562.660048] env[65184]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 562.660048] env[65184]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65184) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 564.663886] env[65184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Session token - acquiring [ 564.664995] env[65184]: DEBUG networking_nsxv3.common.locking [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65184) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 564.664995] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 564.665166] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 564.665166] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 564.674584] env[65184]: DEBUG oslo_concurrency.lockutils [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 564.674838] env[65184]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 564.674978] env[65184]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65184) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 566.680061] env[65184]: CRITICAL neutron [None req-ee445887-c035-49e4-ad59-7a28c8de28b2 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 566.680061] env[65184]: ERROR neutron Traceback (most recent call last): [ 566.680061] env[65184]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 566.680061] env[65184]: ERROR neutron sys.exit(main()) [ 566.680061] env[65184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 566.680061] env[65184]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 566.680061] env[65184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 566.680061] env[65184]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 566.680061] env[65184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 566.680061] env[65184]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 566.680061] env[65184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 566.680061] env[65184]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 566.680061] env[65184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 566.680061] env[65184]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 566.680061] env[65184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 566.680061] env[65184]: ERROR neutron response = self.get(path=path, params=params) [ 566.680958] env[65184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 566.680958] env[65184]: ERROR neutron raise RuntimeError(msg, last_err) [ 566.680958] env[65184]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 566.680958] env[65184]: ERROR neutron [ 569.532613] env[65730]: INFO neutron.common.config [-] Logging enabled! [ 569.532879] env[65730]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 569.533157] env[65730]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65730) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 569.533383] env[65730]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 569.533470] env[65730]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 569.534316] env[65730]: WARNING oslo_policy.policy [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 569.534537] env[65730]: WARNING oslo_policy.policy [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 569.535782] env[65730]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 569.550903] env[65730]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Session token - acquiring [ 569.551784] env[65730]: DEBUG networking_nsxv3.common.locking [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65730) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 569.551995] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 569.552145] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 569.552675] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 569.562213] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 569.562482] env[65730]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 569.562482] env[65730]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65730) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 571.570653] env[65730]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Session token - acquiring [ 571.570653] env[65730]: DEBUG networking_nsxv3.common.locking [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65730) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 571.570653] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 571.571364] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 571.571364] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 571.578854] env[65730]: DEBUG oslo_concurrency.lockutils [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65730) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 571.586374] env[65730]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 571.586374] env[65730]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65730) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 573.580919] env[65730]: CRITICAL neutron [None req-a8821cf6-9bcd-48ea-8946-c7e0ef15a845 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 573.580919] env[65730]: ERROR neutron Traceback (most recent call last): [ 573.580919] env[65730]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 573.580919] env[65730]: ERROR neutron sys.exit(main()) [ 573.580919] env[65730]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 573.580919] env[65730]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 573.580919] env[65730]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 573.580919] env[65730]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 573.580919] env[65730]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 573.580919] env[65730]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 573.580919] env[65730]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 573.580919] env[65730]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 573.580919] env[65730]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 573.580919] env[65730]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 573.580919] env[65730]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 573.580919] env[65730]: ERROR neutron response = self.get(path=path, params=params) [ 573.581693] env[65730]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 573.581693] env[65730]: ERROR neutron raise RuntimeError(msg, last_err) [ 573.581693] env[65730]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 573.581693] env[65730]: ERROR neutron [ 576.440768] env[66021]: INFO neutron.common.config [-] Logging enabled! [ 576.441131] env[66021]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 576.441280] env[66021]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66021) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 576.441470] env[66021]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 576.441567] env[66021]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 576.442434] env[66021]: WARNING oslo_policy.policy [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 576.442672] env[66021]: WARNING oslo_policy.policy [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 576.443891] env[66021]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 576.461170] env[66021]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Session token - acquiring [ 576.462024] env[66021]: DEBUG networking_nsxv3.common.locking [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66021) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 576.462315] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 576.462379] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 576.464191] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 576.476462] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 576.477440] env[66021]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 576.478239] env[66021]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66021) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 578.484440] env[66021]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Session token - acquiring [ 578.484791] env[66021]: DEBUG networking_nsxv3.common.locking [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66021) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 578.484971] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 578.485044] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 578.485424] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 578.495046] env[66021]: DEBUG oslo_concurrency.lockutils [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66021) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 578.495046] env[66021]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 578.495132] env[66021]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66021) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 580.496940] env[66021]: CRITICAL neutron [None req-eac2aa46-a49c-4585-b32e-dd988a945792 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 580.496940] env[66021]: ERROR neutron Traceback (most recent call last): [ 580.496940] env[66021]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 580.496940] env[66021]: ERROR neutron sys.exit(main()) [ 580.496940] env[66021]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 580.496940] env[66021]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 580.496940] env[66021]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 580.496940] env[66021]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 580.496940] env[66021]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 580.496940] env[66021]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 580.496940] env[66021]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 580.496940] env[66021]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 580.496940] env[66021]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 580.496940] env[66021]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 580.496940] env[66021]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 580.496940] env[66021]: ERROR neutron response = self.get(path=path, params=params) [ 580.497787] env[66021]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 580.497787] env[66021]: ERROR neutron raise RuntimeError(msg, last_err) [ 580.497787] env[66021]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 580.497787] env[66021]: ERROR neutron [ 583.557265] env[66494]: INFO neutron.common.config [-] Logging enabled! [ 583.557577] env[66494]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 583.557798] env[66494]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66494) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 583.558041] env[66494]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 583.558137] env[66494]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 583.558982] env[66494]: WARNING oslo_policy.policy [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 583.559248] env[66494]: WARNING oslo_policy.policy [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 583.560487] env[66494]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 583.575351] env[66494]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Session token - acquiring [ 583.576284] env[66494]: DEBUG networking_nsxv3.common.locking [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66494) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 583.576486] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 583.576611] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 583.576985] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 583.586439] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 583.586753] env[66494]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 583.586928] env[66494]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66494) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 585.593293] env[66494]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Session token - acquiring [ 585.593623] env[66494]: DEBUG networking_nsxv3.common.locking [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66494) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 585.593842] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 585.593969] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 585.594243] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 585.605864] env[66494]: DEBUG oslo_concurrency.lockutils [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66494) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 585.606125] env[66494]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 585.606262] env[66494]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66494) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 587.609057] env[66494]: CRITICAL neutron [None req-2adee353-d001-4b91-8db5-66404d1fc4d0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 587.609057] env[66494]: ERROR neutron Traceback (most recent call last): [ 587.609057] env[66494]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 587.609057] env[66494]: ERROR neutron sys.exit(main()) [ 587.609057] env[66494]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 587.609057] env[66494]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 587.609057] env[66494]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 587.609057] env[66494]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 587.609057] env[66494]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 587.609057] env[66494]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 587.609057] env[66494]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 587.609057] env[66494]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 587.609057] env[66494]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 587.609057] env[66494]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 587.609057] env[66494]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 587.609057] env[66494]: ERROR neutron response = self.get(path=path, params=params) [ 587.609861] env[66494]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 587.609861] env[66494]: ERROR neutron raise RuntimeError(msg, last_err) [ 587.609861] env[66494]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 587.609861] env[66494]: ERROR neutron [ 590.343635] env[66737]: INFO neutron.common.config [-] Logging enabled! [ 590.343916] env[66737]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 590.344230] env[66737]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66737) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 590.344437] env[66737]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 590.344528] env[66737]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 590.345368] env[66737]: WARNING oslo_policy.policy [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 590.345579] env[66737]: WARNING oslo_policy.policy [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 590.347915] env[66737]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 590.362781] env[66737]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Session token - acquiring [ 590.363619] env[66737]: DEBUG networking_nsxv3.common.locking [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66737) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 590.363852] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 590.364008] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 590.364407] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 590.373992] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 590.374369] env[66737]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 590.374597] env[66737]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66737) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 592.380832] env[66737]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Session token - acquiring [ 592.381162] env[66737]: DEBUG networking_nsxv3.common.locking [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66737) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 592.381365] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 592.381480] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 592.381768] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 592.391899] env[66737]: DEBUG oslo_concurrency.lockutils [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66737) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 592.392231] env[66737]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 592.392379] env[66737]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66737) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 594.395293] env[66737]: CRITICAL neutron [None req-e39c058e-127a-44d7-983a-ef2d6e9804fa None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 594.395293] env[66737]: ERROR neutron Traceback (most recent call last): [ 594.395293] env[66737]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 594.395293] env[66737]: ERROR neutron sys.exit(main()) [ 594.395293] env[66737]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 594.395293] env[66737]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 594.395293] env[66737]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 594.395293] env[66737]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 594.395293] env[66737]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 594.395293] env[66737]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 594.395293] env[66737]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 594.395293] env[66737]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 594.395293] env[66737]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 594.395293] env[66737]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 594.395293] env[66737]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 594.395293] env[66737]: ERROR neutron response = self.get(path=path, params=params) [ 594.395993] env[66737]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 594.395993] env[66737]: ERROR neutron raise RuntimeError(msg, last_err) [ 594.395993] env[66737]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 594.395993] env[66737]: ERROR neutron [ 597.055776] env[66967]: INFO neutron.common.config [-] Logging enabled! [ 597.056099] env[66967]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 597.056380] env[66967]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66967) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 597.056604] env[66967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 597.056700] env[66967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 597.057564] env[66967]: WARNING oslo_policy.policy [None req-25578125-44df-403c-9628-451357410a49 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 597.057781] env[66967]: WARNING oslo_policy.policy [None req-25578125-44df-403c-9628-451357410a49 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 597.058996] env[66967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-25578125-44df-403c-9628-451357410a49 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 597.074007] env[66967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25578125-44df-403c-9628-451357410a49 None None] Session token - acquiring [ 597.074845] env[66967]: DEBUG networking_nsxv3.common.locking [None req-25578125-44df-403c-9628-451357410a49 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66967) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 597.075049] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 597.075176] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 597.075543] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 597.098186] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 597.098500] env[66967]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25578125-44df-403c-9628-451357410a49 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 597.098651] env[66967]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25578125-44df-403c-9628-451357410a49 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66967) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 599.104647] env[66967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25578125-44df-403c-9628-451357410a49 None None] Session token - acquiring [ 599.105068] env[66967]: DEBUG networking_nsxv3.common.locking [None req-25578125-44df-403c-9628-451357410a49 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66967) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 599.105146] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 599.105245] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 599.105533] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 599.115872] env[66967]: DEBUG oslo_concurrency.lockutils [None req-25578125-44df-403c-9628-451357410a49 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 599.115872] env[66967]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25578125-44df-403c-9628-451357410a49 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 599.115872] env[66967]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25578125-44df-403c-9628-451357410a49 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66967) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 601.121624] env[66967]: CRITICAL neutron [None req-25578125-44df-403c-9628-451357410a49 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 601.121624] env[66967]: ERROR neutron Traceback (most recent call last): [ 601.121624] env[66967]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 601.121624] env[66967]: ERROR neutron sys.exit(main()) [ 601.121624] env[66967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 601.121624] env[66967]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 601.121624] env[66967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 601.121624] env[66967]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 601.121624] env[66967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 601.121624] env[66967]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 601.121624] env[66967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 601.121624] env[66967]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 601.121624] env[66967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 601.121624] env[66967]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 601.121624] env[66967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 601.121624] env[66967]: ERROR neutron response = self.get(path=path, params=params) [ 601.122282] env[66967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 601.122282] env[66967]: ERROR neutron raise RuntimeError(msg, last_err) [ 601.122282] env[66967]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 601.122282] env[66967]: ERROR neutron [ 603.881158] env[67076]: INFO neutron.common.config [-] Logging enabled! [ 603.881381] env[67076]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 603.881716] env[67076]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67076) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 603.881933] env[67076]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 603.882028] env[67076]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 603.882872] env[67076]: WARNING oslo_policy.policy [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 603.883100] env[67076]: WARNING oslo_policy.policy [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 603.884344] env[67076]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 603.899386] env[67076]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Session token - acquiring [ 603.900218] env[67076]: DEBUG networking_nsxv3.common.locking [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67076) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 603.900438] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 603.900539] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 603.900909] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 603.910859] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 603.911108] env[67076]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 603.911249] env[67076]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67076) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 605.919968] env[67076]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Session token - acquiring [ 605.919968] env[67076]: DEBUG networking_nsxv3.common.locking [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67076) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 605.919968] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 605.920358] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 605.920358] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 605.929418] env[67076]: DEBUG oslo_concurrency.lockutils [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67076) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 605.929852] env[67076]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 605.930134] env[67076]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67076) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 607.932914] env[67076]: CRITICAL neutron [None req-dffaf0af-eeae-4bce-be07-d9c85f900174 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 607.932914] env[67076]: ERROR neutron Traceback (most recent call last): [ 607.932914] env[67076]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 607.932914] env[67076]: ERROR neutron sys.exit(main()) [ 607.932914] env[67076]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 607.932914] env[67076]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 607.932914] env[67076]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 607.932914] env[67076]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 607.932914] env[67076]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 607.932914] env[67076]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 607.932914] env[67076]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 607.932914] env[67076]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 607.932914] env[67076]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 607.932914] env[67076]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 607.932914] env[67076]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 607.932914] env[67076]: ERROR neutron response = self.get(path=path, params=params) [ 607.933783] env[67076]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 607.933783] env[67076]: ERROR neutron raise RuntimeError(msg, last_err) [ 607.933783] env[67076]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 607.933783] env[67076]: ERROR neutron [ 610.681366] env[67405]: INFO neutron.common.config [-] Logging enabled! [ 610.681626] env[67405]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 610.681924] env[67405]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67405) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 610.682171] env[67405]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 610.682269] env[67405]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 610.683137] env[67405]: WARNING oslo_policy.policy [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 610.683369] env[67405]: WARNING oslo_policy.policy [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 610.684708] env[67405]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 610.717928] env[67405]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Session token - acquiring [ 610.723014] env[67405]: DEBUG networking_nsxv3.common.locking [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67405) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 610.723291] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 610.723455] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 610.724086] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 610.734157] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 610.734418] env[67405]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 610.734566] env[67405]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67405) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 612.742795] env[67405]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Session token - acquiring [ 612.742795] env[67405]: DEBUG networking_nsxv3.common.locking [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67405) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 612.742795] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 612.743111] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 612.743111] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 612.751471] env[67405]: DEBUG oslo_concurrency.lockutils [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67405) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 612.751819] env[67405]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 612.752061] env[67405]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67405) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 614.764024] env[67405]: CRITICAL neutron [None req-5362e252-c2cd-4659-966a-c070f466d6ec None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 614.764024] env[67405]: ERROR neutron Traceback (most recent call last): [ 614.764024] env[67405]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 614.764024] env[67405]: ERROR neutron sys.exit(main()) [ 614.764024] env[67405]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 614.764024] env[67405]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 614.764024] env[67405]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 614.764024] env[67405]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 614.764024] env[67405]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 614.764024] env[67405]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 614.764024] env[67405]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 614.764024] env[67405]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 614.764024] env[67405]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 614.764024] env[67405]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 614.764024] env[67405]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 614.764024] env[67405]: ERROR neutron response = self.get(path=path, params=params) [ 614.764853] env[67405]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 614.764853] env[67405]: ERROR neutron raise RuntimeError(msg, last_err) [ 614.764853] env[67405]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 614.764853] env[67405]: ERROR neutron [ 617.478783] env[67595]: INFO neutron.common.config [-] Logging enabled! [ 617.479104] env[67595]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 617.479317] env[67595]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67595) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 617.479547] env[67595]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 617.479641] env[67595]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 617.480503] env[67595]: WARNING oslo_policy.policy [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 617.480721] env[67595]: WARNING oslo_policy.policy [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 617.482014] env[67595]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 617.498904] env[67595]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Session token - acquiring [ 617.498904] env[67595]: DEBUG networking_nsxv3.common.locking [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67595) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 617.498904] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 617.499081] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 617.499081] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 617.508044] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 617.508286] env[67595]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 617.508430] env[67595]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67595) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 619.514514] env[67595]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Session token - acquiring [ 619.514992] env[67595]: DEBUG networking_nsxv3.common.locking [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67595) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 619.514992] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 619.515116] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 619.515385] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 619.525775] env[67595]: DEBUG oslo_concurrency.lockutils [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67595) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 619.526005] env[67595]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 619.526250] env[67595]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67595) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 621.528915] env[67595]: CRITICAL neutron [None req-3a9c6d31-669a-46e1-b0ec-b382dff31b69 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 621.528915] env[67595]: ERROR neutron Traceback (most recent call last): [ 621.528915] env[67595]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 621.528915] env[67595]: ERROR neutron sys.exit(main()) [ 621.528915] env[67595]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 621.528915] env[67595]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 621.528915] env[67595]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 621.528915] env[67595]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 621.528915] env[67595]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 621.528915] env[67595]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 621.528915] env[67595]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 621.528915] env[67595]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 621.528915] env[67595]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 621.528915] env[67595]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 621.528915] env[67595]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 621.528915] env[67595]: ERROR neutron response = self.get(path=path, params=params) [ 621.529704] env[67595]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 621.529704] env[67595]: ERROR neutron raise RuntimeError(msg, last_err) [ 621.529704] env[67595]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 621.529704] env[67595]: ERROR neutron [ 624.284038] env[67680]: INFO neutron.common.config [-] Logging enabled! [ 624.284362] env[67680]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 624.284625] env[67680]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67680) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 624.284855] env[67680]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 624.284964] env[67680]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 624.285857] env[67680]: WARNING oslo_policy.policy [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 624.286113] env[67680]: WARNING oslo_policy.policy [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 624.287443] env[67680]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 624.302367] env[67680]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Session token - acquiring [ 624.303223] env[67680]: DEBUG networking_nsxv3.common.locking [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67680) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 624.303432] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 624.303562] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 624.303947] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 624.313362] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 624.313637] env[67680]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 624.313787] env[67680]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67680) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 626.322636] env[67680]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Session token - acquiring [ 626.322962] env[67680]: DEBUG networking_nsxv3.common.locking [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67680) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 626.323110] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 626.323232] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 626.323511] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 626.332679] env[67680]: DEBUG oslo_concurrency.lockutils [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67680) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 626.332909] env[67680]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 626.333054] env[67680]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67680) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 628.344138] env[67680]: CRITICAL neutron [None req-aa3072a0-cf81-48ca-96fd-df5f56567cea None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 628.344138] env[67680]: ERROR neutron Traceback (most recent call last): [ 628.344138] env[67680]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 628.344138] env[67680]: ERROR neutron sys.exit(main()) [ 628.344138] env[67680]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 628.344138] env[67680]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 628.344138] env[67680]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 628.344138] env[67680]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 628.344138] env[67680]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 628.344138] env[67680]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 628.344138] env[67680]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 628.344138] env[67680]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 628.344138] env[67680]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 628.344138] env[67680]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 628.344138] env[67680]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 628.344138] env[67680]: ERROR neutron response = self.get(path=path, params=params) [ 628.344997] env[67680]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 628.344997] env[67680]: ERROR neutron raise RuntimeError(msg, last_err) [ 628.344997] env[67680]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 628.344997] env[67680]: ERROR neutron [ 631.014534] env[67995]: INFO neutron.common.config [-] Logging enabled! [ 631.014878] env[67995]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 631.015076] env[67995]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67995) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 631.015308] env[67995]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 631.015398] env[67995]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 631.016301] env[67995]: WARNING oslo_policy.policy [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 631.016511] env[67995]: WARNING oslo_policy.policy [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 631.017720] env[67995]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 631.042448] env[67995]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Session token - acquiring [ 631.043270] env[67995]: DEBUG networking_nsxv3.common.locking [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67995) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 631.043702] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 631.043702] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 631.044033] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 631.053859] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 631.054198] env[67995]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 631.054404] env[67995]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67995) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 633.060995] env[67995]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Session token - acquiring [ 633.061321] env[67995]: DEBUG networking_nsxv3.common.locking [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67995) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 633.061532] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 633.061662] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 633.061935] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 633.071392] env[67995]: DEBUG oslo_concurrency.lockutils [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67995) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 633.071666] env[67995]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 633.071806] env[67995]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67995) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 635.072884] env[67995]: CRITICAL neutron [None req-d55bce15-42bd-46bd-9952-fefea7c3e274 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 635.072884] env[67995]: ERROR neutron Traceback (most recent call last): [ 635.072884] env[67995]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 635.072884] env[67995]: ERROR neutron sys.exit(main()) [ 635.072884] env[67995]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 635.072884] env[67995]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 635.072884] env[67995]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 635.072884] env[67995]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 635.072884] env[67995]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 635.072884] env[67995]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 635.072884] env[67995]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 635.072884] env[67995]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 635.072884] env[67995]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 635.072884] env[67995]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 635.072884] env[67995]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 635.072884] env[67995]: ERROR neutron response = self.get(path=path, params=params) [ 635.073565] env[67995]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 635.073565] env[67995]: ERROR neutron raise RuntimeError(msg, last_err) [ 635.073565] env[67995]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 635.073565] env[67995]: ERROR neutron [ 637.773126] env[68045]: INFO neutron.common.config [-] Logging enabled! [ 637.773470] env[68045]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 637.773615] env[68045]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68045) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 637.773844] env[68045]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 637.773932] env[68045]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 637.774895] env[68045]: WARNING oslo_policy.policy [None req-60408934-8166-4f4e-affd-423925450dd0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 637.774985] env[68045]: WARNING oslo_policy.policy [None req-60408934-8166-4f4e-affd-423925450dd0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 637.776225] env[68045]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 637.793191] env[68045]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Session token - acquiring [ 637.794014] env[68045]: DEBUG networking_nsxv3.common.locking [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68045) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 637.794228] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 637.794358] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 637.794736] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 637.804691] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 637.805543] env[68045]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 637.805543] env[68045]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68045) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 639.811728] env[68045]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Session token - acquiring [ 639.812058] env[68045]: DEBUG networking_nsxv3.common.locking [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68045) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 639.812252] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 639.812375] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 639.812652] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 639.821453] env[68045]: DEBUG oslo_concurrency.lockutils [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68045) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 639.821704] env[68045]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 639.821828] env[68045]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68045) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 641.824762] env[68045]: CRITICAL neutron [None req-60408934-8166-4f4e-affd-423925450dd0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 641.824762] env[68045]: ERROR neutron Traceback (most recent call last): [ 641.824762] env[68045]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 641.824762] env[68045]: ERROR neutron sys.exit(main()) [ 641.824762] env[68045]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 641.824762] env[68045]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 641.824762] env[68045]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 641.824762] env[68045]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 641.824762] env[68045]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 641.824762] env[68045]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 641.824762] env[68045]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 641.824762] env[68045]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 641.824762] env[68045]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 641.824762] env[68045]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 641.824762] env[68045]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 641.824762] env[68045]: ERROR neutron response = self.get(path=path, params=params) [ 641.825455] env[68045]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 641.825455] env[68045]: ERROR neutron raise RuntimeError(msg, last_err) [ 641.825455] env[68045]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 641.825455] env[68045]: ERROR neutron [ 644.519658] env[68112]: INFO neutron.common.config [-] Logging enabled! [ 644.519960] env[68112]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 644.520268] env[68112]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68112) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 644.520505] env[68112]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 644.520601] env[68112]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 644.521460] env[68112]: WARNING oslo_policy.policy [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 644.521681] env[68112]: WARNING oslo_policy.policy [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 644.522895] env[68112]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 644.537092] env[68112]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Session token - acquiring [ 644.537925] env[68112]: DEBUG networking_nsxv3.common.locking [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68112) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 644.538136] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 644.538292] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 644.538668] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 644.553060] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 644.553361] env[68112]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 644.553518] env[68112]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68112) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 646.559722] env[68112]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Session token - acquiring [ 646.560360] env[68112]: DEBUG networking_nsxv3.common.locking [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68112) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 646.560360] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 646.560464] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 646.560661] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 646.569812] env[68112]: DEBUG oslo_concurrency.lockutils [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68112) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 646.570065] env[68112]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 646.570208] env[68112]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68112) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 648.573128] env[68112]: CRITICAL neutron [None req-6710b99d-ca3a-4302-b109-0df2f424e9f4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 648.573128] env[68112]: ERROR neutron Traceback (most recent call last): [ 648.573128] env[68112]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 648.573128] env[68112]: ERROR neutron sys.exit(main()) [ 648.573128] env[68112]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 648.573128] env[68112]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 648.573128] env[68112]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 648.573128] env[68112]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 648.573128] env[68112]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 648.573128] env[68112]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 648.573128] env[68112]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 648.573128] env[68112]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 648.573128] env[68112]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 648.573128] env[68112]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 648.573128] env[68112]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 648.573128] env[68112]: ERROR neutron response = self.get(path=path, params=params) [ 648.574218] env[68112]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 648.574218] env[68112]: ERROR neutron raise RuntimeError(msg, last_err) [ 648.574218] env[68112]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 648.574218] env[68112]: ERROR neutron [ 651.136574] env[68186]: INFO neutron.common.config [-] Logging enabled! [ 651.136987] env[68186]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 651.137190] env[68186]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68186) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 651.137414] env[68186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 651.137514] env[68186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 651.138482] env[68186]: WARNING oslo_policy.policy [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 651.138569] env[68186]: WARNING oslo_policy.policy [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 651.139775] env[68186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 651.153906] env[68186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Session token - acquiring [ 651.154673] env[68186]: DEBUG networking_nsxv3.common.locking [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68186) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 651.154874] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 651.154998] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 651.155352] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 651.164690] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 651.164892] env[68186]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 651.165124] env[68186]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68186) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 653.170967] env[68186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Session token - acquiring [ 653.171283] env[68186]: DEBUG networking_nsxv3.common.locking [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68186) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 653.171685] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 653.171685] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 653.171857] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 653.181603] env[68186]: DEBUG oslo_concurrency.lockutils [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 653.181863] env[68186]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 653.182266] env[68186]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68186) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 655.185209] env[68186]: CRITICAL neutron [None req-b0011669-31ce-4f24-8d3f-4f2627016b96 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 655.185209] env[68186]: ERROR neutron Traceback (most recent call last): [ 655.185209] env[68186]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 655.185209] env[68186]: ERROR neutron sys.exit(main()) [ 655.185209] env[68186]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 655.185209] env[68186]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 655.185209] env[68186]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 655.185209] env[68186]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 655.185209] env[68186]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 655.185209] env[68186]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 655.185209] env[68186]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 655.185209] env[68186]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 655.185209] env[68186]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 655.185209] env[68186]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 655.185209] env[68186]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 655.185209] env[68186]: ERROR neutron response = self.get(path=path, params=params) [ 655.185978] env[68186]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 655.185978] env[68186]: ERROR neutron raise RuntimeError(msg, last_err) [ 655.185978] env[68186]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 655.185978] env[68186]: ERROR neutron [ 657.982068] env[68276]: INFO neutron.common.config [-] Logging enabled! [ 657.982325] env[68276]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 657.982626] env[68276]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68276) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 657.982816] env[68276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 657.982905] env[68276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 657.983783] env[68276]: WARNING oslo_policy.policy [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 657.984188] env[68276]: WARNING oslo_policy.policy [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 657.985297] env[68276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 658.008779] env[68276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Session token - acquiring [ 658.011408] env[68276]: DEBUG networking_nsxv3.common.locking [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68276) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 658.011547] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 658.011679] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 658.012111] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 658.033099] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 658.033381] env[68276]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 658.033526] env[68276]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68276) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 660.041169] env[68276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Session token - acquiring [ 660.041553] env[68276]: DEBUG networking_nsxv3.common.locking [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68276) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 660.041670] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 660.041766] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 660.042073] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 660.051499] env[68276]: DEBUG oslo_concurrency.lockutils [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 660.051743] env[68276]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 660.051880] env[68276]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68276) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 662.053221] env[68276]: CRITICAL neutron [None req-0ae274f6-f900-41df-a429-ff7f674c286b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 662.053221] env[68276]: ERROR neutron Traceback (most recent call last): [ 662.053221] env[68276]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 662.053221] env[68276]: ERROR neutron sys.exit(main()) [ 662.053221] env[68276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 662.053221] env[68276]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 662.053221] env[68276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 662.053221] env[68276]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 662.053221] env[68276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 662.053221] env[68276]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 662.053221] env[68276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 662.053221] env[68276]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 662.053221] env[68276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 662.053221] env[68276]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 662.053221] env[68276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 662.053221] env[68276]: ERROR neutron response = self.get(path=path, params=params) [ 662.053846] env[68276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 662.053846] env[68276]: ERROR neutron raise RuntimeError(msg, last_err) [ 662.053846] env[68276]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 662.053846] env[68276]: ERROR neutron [ 664.774344] env[68638]: INFO neutron.common.config [-] Logging enabled! [ 664.774586] env[68638]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 664.775246] env[68638]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68638) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 664.775246] env[68638]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 664.775246] env[68638]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 664.779767] env[68638]: WARNING oslo_policy.policy [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 664.779767] env[68638]: WARNING oslo_policy.policy [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 664.779767] env[68638]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 664.800617] env[68638]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Session token - acquiring [ 664.801542] env[68638]: DEBUG networking_nsxv3.common.locking [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68638) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 664.801757] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 664.801885] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 664.802264] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 664.811817] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 664.812057] env[68638]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 664.812209] env[68638]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68638) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 666.820123] env[68638]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Session token - acquiring [ 666.820123] env[68638]: DEBUG networking_nsxv3.common.locking [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68638) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 666.820487] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 666.820487] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 666.820729] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 666.830472] env[68638]: DEBUG oslo_concurrency.lockutils [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68638) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 666.830713] env[68638]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 666.830853] env[68638]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68638) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 668.836046] env[68638]: CRITICAL neutron [None req-51391608-ae66-46c0-90dd-8cbd1af69167 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 668.836046] env[68638]: ERROR neutron Traceback (most recent call last): [ 668.836046] env[68638]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 668.836046] env[68638]: ERROR neutron sys.exit(main()) [ 668.836046] env[68638]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 668.836046] env[68638]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 668.836046] env[68638]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 668.836046] env[68638]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 668.836046] env[68638]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 668.836046] env[68638]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 668.836046] env[68638]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 668.836046] env[68638]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 668.836046] env[68638]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 668.836046] env[68638]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 668.836046] env[68638]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 668.836046] env[68638]: ERROR neutron response = self.get(path=path, params=params) [ 668.836975] env[68638]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 668.836975] env[68638]: ERROR neutron raise RuntimeError(msg, last_err) [ 668.836975] env[68638]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 668.836975] env[68638]: ERROR neutron [ 671.630133] env[69004]: INFO neutron.common.config [-] Logging enabled! [ 671.630393] env[69004]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 671.631061] env[69004]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69004) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 671.631061] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 671.631061] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 671.631823] env[69004]: WARNING oslo_policy.policy [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 671.633890] env[69004]: WARNING oslo_policy.policy [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 671.634937] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 671.650753] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Session token - acquiring [ 671.652590] env[69004]: DEBUG networking_nsxv3.common.locking [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69004) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 671.652834] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 671.652965] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 671.653346] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 671.663354] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 671.663354] env[69004]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 671.663354] env[69004]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69004) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 673.669426] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Session token - acquiring [ 673.669803] env[69004]: DEBUG networking_nsxv3.common.locking [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69004) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 673.669993] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 673.670119] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 673.670393] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 673.680393] env[69004]: DEBUG oslo_concurrency.lockutils [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 673.680657] env[69004]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 673.680802] env[69004]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69004) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 675.683692] env[69004]: CRITICAL neutron [None req-2f896837-b1d4-49f2-8cc1-65dabba92e7d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 675.683692] env[69004]: ERROR neutron Traceback (most recent call last): [ 675.683692] env[69004]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 675.683692] env[69004]: ERROR neutron sys.exit(main()) [ 675.683692] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 675.683692] env[69004]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 675.683692] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 675.683692] env[69004]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 675.683692] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 675.683692] env[69004]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 675.683692] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 675.683692] env[69004]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 675.683692] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 675.683692] env[69004]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 675.683692] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 675.683692] env[69004]: ERROR neutron response = self.get(path=path, params=params) [ 675.684843] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 675.684843] env[69004]: ERROR neutron raise RuntimeError(msg, last_err) [ 675.684843] env[69004]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 675.684843] env[69004]: ERROR neutron [ 678.582927] env[69489]: INFO neutron.common.config [-] Logging enabled! [ 678.583250] env[69489]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 678.583495] env[69489]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69489) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 678.583740] env[69489]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 678.583833] env[69489]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 678.584738] env[69489]: WARNING oslo_policy.policy [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 678.584983] env[69489]: WARNING oslo_policy.policy [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 678.586303] env[69489]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 678.601253] env[69489]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Session token - acquiring [ 678.602230] env[69489]: DEBUG networking_nsxv3.common.locking [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69489) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 678.602433] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 678.602559] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 678.602930] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 678.612262] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 678.612474] env[69489]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 678.612616] env[69489]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69489) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 680.620480] env[69489]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Session token - acquiring [ 680.621079] env[69489]: DEBUG networking_nsxv3.common.locking [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69489) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 680.621351] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 680.621488] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 680.621816] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 680.640818] env[69489]: DEBUG oslo_concurrency.lockutils [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69489) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 680.641079] env[69489]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 680.641215] env[69489]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69489) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 682.644156] env[69489]: CRITICAL neutron [None req-0170d914-bd7c-4080-a2a7-2e5aade6bd92 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 682.644156] env[69489]: ERROR neutron Traceback (most recent call last): [ 682.644156] env[69489]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 682.644156] env[69489]: ERROR neutron sys.exit(main()) [ 682.644156] env[69489]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 682.644156] env[69489]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 682.644156] env[69489]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 682.644156] env[69489]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 682.644156] env[69489]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 682.644156] env[69489]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 682.644156] env[69489]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 682.644156] env[69489]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 682.644156] env[69489]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 682.644156] env[69489]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 682.644156] env[69489]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 682.644156] env[69489]: ERROR neutron response = self.get(path=path, params=params) [ 682.644836] env[69489]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 682.644836] env[69489]: ERROR neutron raise RuntimeError(msg, last_err) [ 682.644836] env[69489]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 682.644836] env[69489]: ERROR neutron [ 685.345820] env[69596]: INFO neutron.common.config [-] Logging enabled! [ 685.345820] env[69596]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 685.346156] env[69596]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69596) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 685.347471] env[69596]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 685.347471] env[69596]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 685.347630] env[69596]: WARNING oslo_policy.policy [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 685.348173] env[69596]: WARNING oslo_policy.policy [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 685.349136] env[69596]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 685.363911] env[69596]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Session token - acquiring [ 685.364851] env[69596]: DEBUG networking_nsxv3.common.locking [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69596) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 685.365137] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 685.365322] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 685.365734] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 685.375700] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 685.376126] env[69596]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 685.376344] env[69596]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69596) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 687.382362] env[69596]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Session token - acquiring [ 687.382668] env[69596]: DEBUG networking_nsxv3.common.locking [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69596) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 687.382832] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 687.382958] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 687.383246] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 687.392536] env[69596]: DEBUG oslo_concurrency.lockutils [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69596) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 687.392781] env[69596]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 687.392974] env[69596]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69596) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 689.395801] env[69596]: CRITICAL neutron [None req-36375087-8ca5-453e-a7c0-b3cea2b0027b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 689.395801] env[69596]: ERROR neutron Traceback (most recent call last): [ 689.395801] env[69596]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 689.395801] env[69596]: ERROR neutron sys.exit(main()) [ 689.395801] env[69596]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 689.395801] env[69596]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 689.395801] env[69596]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 689.395801] env[69596]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 689.395801] env[69596]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 689.395801] env[69596]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 689.395801] env[69596]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 689.395801] env[69596]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 689.395801] env[69596]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 689.395801] env[69596]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 689.395801] env[69596]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 689.395801] env[69596]: ERROR neutron response = self.get(path=path, params=params) [ 689.396480] env[69596]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 689.396480] env[69596]: ERROR neutron raise RuntimeError(msg, last_err) [ 689.396480] env[69596]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 689.396480] env[69596]: ERROR neutron [ 692.177235] env[69715]: INFO neutron.common.config [-] Logging enabled! [ 692.177581] env[69715]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 692.177827] env[69715]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69715) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 692.178079] env[69715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 692.178180] env[69715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 692.179116] env[69715]: WARNING oslo_policy.policy [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 692.179344] env[69715]: WARNING oslo_policy.policy [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 692.180841] env[69715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 692.197233] env[69715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Session token - acquiring [ 692.198176] env[69715]: DEBUG networking_nsxv3.common.locking [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69715) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 692.198362] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 692.198483] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 692.198865] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 692.211287] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 692.211287] env[69715]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 692.211287] env[69715]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69715) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 694.214883] env[69715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Session token - acquiring [ 694.215140] env[69715]: DEBUG networking_nsxv3.common.locking [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69715) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 694.215693] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 694.215693] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 694.215693] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 694.225275] env[69715]: DEBUG oslo_concurrency.lockutils [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 694.225685] env[69715]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 694.226005] env[69715]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69715) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 696.228885] env[69715]: CRITICAL neutron [None req-d6605e7a-7c4f-488a-827f-1e6693690832 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 696.228885] env[69715]: ERROR neutron Traceback (most recent call last): [ 696.228885] env[69715]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 696.228885] env[69715]: ERROR neutron sys.exit(main()) [ 696.228885] env[69715]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 696.228885] env[69715]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 696.228885] env[69715]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 696.228885] env[69715]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 696.228885] env[69715]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 696.228885] env[69715]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 696.228885] env[69715]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 696.228885] env[69715]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 696.228885] env[69715]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 696.228885] env[69715]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 696.228885] env[69715]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 696.228885] env[69715]: ERROR neutron response = self.get(path=path, params=params) [ 696.229698] env[69715]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 696.229698] env[69715]: ERROR neutron raise RuntimeError(msg, last_err) [ 696.229698] env[69715]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 696.229698] env[69715]: ERROR neutron [ 698.984593] env[69963]: INFO neutron.common.config [-] Logging enabled! [ 698.984914] env[69963]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 698.985171] env[69963]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69963) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 698.985421] env[69963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 698.985517] env[69963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 698.986482] env[69963]: WARNING oslo_policy.policy [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 698.986727] env[69963]: WARNING oslo_policy.policy [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 698.987979] env[69963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 699.002733] env[69963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Session token - acquiring [ 699.003582] env[69963]: DEBUG networking_nsxv3.common.locking [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69963) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 699.003883] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 699.003960] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 699.004348] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 699.022370] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 699.022666] env[69963]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 699.022815] env[69963]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69963) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 701.029116] env[69963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Session token - acquiring [ 701.029463] env[69963]: DEBUG networking_nsxv3.common.locking [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69963) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 701.029788] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 701.029788] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 701.030029] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 701.040374] env[69963]: DEBUG oslo_concurrency.lockutils [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 701.040616] env[69963]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 701.040751] env[69963]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69963) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 703.042104] env[69963]: CRITICAL neutron [None req-9c819bc5-ee82-4708-b89b-cb732edb7b89 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 703.042104] env[69963]: ERROR neutron Traceback (most recent call last): [ 703.042104] env[69963]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 703.042104] env[69963]: ERROR neutron sys.exit(main()) [ 703.042104] env[69963]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 703.042104] env[69963]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 703.042104] env[69963]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 703.042104] env[69963]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 703.042104] env[69963]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 703.042104] env[69963]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 703.042104] env[69963]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 703.042104] env[69963]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 703.042104] env[69963]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 703.042104] env[69963]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 703.042104] env[69963]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 703.042104] env[69963]: ERROR neutron response = self.get(path=path, params=params) [ 703.042880] env[69963]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 703.042880] env[69963]: ERROR neutron raise RuntimeError(msg, last_err) [ 703.042880] env[69963]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 703.042880] env[69963]: ERROR neutron [ 705.840628] env[70287]: INFO neutron.common.config [-] Logging enabled! [ 705.840952] env[70287]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 705.841209] env[70287]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70287) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 705.841447] env[70287]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 705.841543] env[70287]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 705.842415] env[70287]: WARNING oslo_policy.policy [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 705.842599] env[70287]: WARNING oslo_policy.policy [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 705.844792] env[70287]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 705.858142] env[70287]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Session token - acquiring [ 705.858943] env[70287]: DEBUG networking_nsxv3.common.locking [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70287) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 705.859244] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 705.859319] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 705.859630] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 705.869003] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 705.869269] env[70287]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 705.869417] env[70287]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70287) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 707.875761] env[70287]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Session token - acquiring [ 707.876492] env[70287]: DEBUG networking_nsxv3.common.locking [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70287) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 707.876492] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 707.876898] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 707.877310] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 707.886626] env[70287]: DEBUG oslo_concurrency.lockutils [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70287) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 707.886892] env[70287]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 707.887221] env[70287]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70287) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 709.889417] env[70287]: CRITICAL neutron [None req-1feee38b-78e3-49f4-b42c-b5527b735c93 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 709.889417] env[70287]: ERROR neutron Traceback (most recent call last): [ 709.889417] env[70287]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 709.889417] env[70287]: ERROR neutron sys.exit(main()) [ 709.889417] env[70287]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 709.889417] env[70287]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 709.889417] env[70287]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 709.889417] env[70287]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 709.889417] env[70287]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 709.889417] env[70287]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 709.889417] env[70287]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 709.889417] env[70287]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 709.889417] env[70287]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 709.889417] env[70287]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 709.889417] env[70287]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 709.889417] env[70287]: ERROR neutron response = self.get(path=path, params=params) [ 709.891053] env[70287]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 709.891053] env[70287]: ERROR neutron raise RuntimeError(msg, last_err) [ 709.891053] env[70287]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 709.891053] env[70287]: ERROR neutron [ 712.702535] env[70576]: INFO neutron.common.config [-] Logging enabled! [ 712.703031] env[70576]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 712.703147] env[70576]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70576) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 712.703369] env[70576]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 712.703585] env[70576]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 712.704443] env[70576]: WARNING oslo_policy.policy [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 712.704550] env[70576]: WARNING oslo_policy.policy [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 712.705815] env[70576]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 712.721779] env[70576]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Session token - acquiring [ 712.722650] env[70576]: DEBUG networking_nsxv3.common.locking [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70576) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 712.723183] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 712.723183] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 712.723408] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 712.733087] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 712.733390] env[70576]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 712.733540] env[70576]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70576) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 714.740028] env[70576]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Session token - acquiring [ 714.740336] env[70576]: DEBUG networking_nsxv3.common.locking [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70576) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 714.740336] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 714.740444] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 714.740679] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 714.750685] env[70576]: DEBUG oslo_concurrency.lockutils [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70576) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 714.750944] env[70576]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 714.751223] env[70576]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70576) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 716.755339] env[70576]: CRITICAL neutron [None req-139dc493-d06c-48e6-b2fc-0dfac17162c2 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 716.755339] env[70576]: ERROR neutron Traceback (most recent call last): [ 716.755339] env[70576]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 716.755339] env[70576]: ERROR neutron sys.exit(main()) [ 716.755339] env[70576]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 716.755339] env[70576]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 716.755339] env[70576]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 716.755339] env[70576]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 716.755339] env[70576]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 716.755339] env[70576]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 716.755339] env[70576]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 716.755339] env[70576]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 716.755339] env[70576]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 716.755339] env[70576]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 716.755339] env[70576]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 716.755339] env[70576]: ERROR neutron response = self.get(path=path, params=params) [ 716.756436] env[70576]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 716.756436] env[70576]: ERROR neutron raise RuntimeError(msg, last_err) [ 716.756436] env[70576]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 716.756436] env[70576]: ERROR neutron [ 719.597219] env[70950]: INFO neutron.common.config [-] Logging enabled! [ 719.597563] env[70950]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 719.597724] env[70950]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70950) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 719.597941] env[70950]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 719.598029] env[70950]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 719.598860] env[70950]: WARNING oslo_policy.policy [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 719.599070] env[70950]: WARNING oslo_policy.policy [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 719.600301] env[70950]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 719.614323] env[70950]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Session token - acquiring [ 719.615113] env[70950]: DEBUG networking_nsxv3.common.locking [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70950) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 719.615313] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 719.615436] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 719.615795] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 719.624920] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 719.625159] env[70950]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 719.625363] env[70950]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70950) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 721.631499] env[70950]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Session token - acquiring [ 721.631812] env[70950]: DEBUG networking_nsxv3.common.locking [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70950) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 721.632046] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 721.632184] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 721.632499] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 721.641903] env[70950]: DEBUG oslo_concurrency.lockutils [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70950) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 721.642149] env[70950]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 721.642317] env[70950]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70950) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 723.644887] env[70950]: CRITICAL neutron [None req-e5c494b2-77b3-426e-be89-cbb22c722366 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 723.644887] env[70950]: ERROR neutron Traceback (most recent call last): [ 723.644887] env[70950]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 723.644887] env[70950]: ERROR neutron sys.exit(main()) [ 723.644887] env[70950]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 723.644887] env[70950]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 723.644887] env[70950]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 723.644887] env[70950]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 723.644887] env[70950]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 723.644887] env[70950]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 723.644887] env[70950]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 723.644887] env[70950]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 723.644887] env[70950]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 723.644887] env[70950]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 723.644887] env[70950]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 723.644887] env[70950]: ERROR neutron response = self.get(path=path, params=params) [ 723.645694] env[70950]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 723.645694] env[70950]: ERROR neutron raise RuntimeError(msg, last_err) [ 723.645694] env[70950]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 723.645694] env[70950]: ERROR neutron [ 726.281422] env[71191]: INFO neutron.common.config [-] Logging enabled! [ 726.281765] env[71191]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 726.281946] env[71191]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71191) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 726.282190] env[71191]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 726.282285] env[71191]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 726.283146] env[71191]: WARNING oslo_policy.policy [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 726.283395] env[71191]: WARNING oslo_policy.policy [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 726.284656] env[71191]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 726.299837] env[71191]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Session token - acquiring [ 726.300630] env[71191]: DEBUG networking_nsxv3.common.locking [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71191) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 726.300858] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 726.300991] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 726.301353] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 726.311043] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 726.311293] env[71191]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 726.311440] env[71191]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71191) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 728.317503] env[71191]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Session token - acquiring [ 728.317839] env[71191]: DEBUG networking_nsxv3.common.locking [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71191) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 728.318036] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 728.318162] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 728.318441] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 728.327645] env[71191]: DEBUG oslo_concurrency.lockutils [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71191) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 728.328247] env[71191]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 728.328401] env[71191]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71191) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 730.332654] env[71191]: CRITICAL neutron [None req-bdd89521-2972-4746-a7d1-93d117c1a57a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 730.332654] env[71191]: ERROR neutron Traceback (most recent call last): [ 730.332654] env[71191]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 730.332654] env[71191]: ERROR neutron sys.exit(main()) [ 730.332654] env[71191]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 730.332654] env[71191]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 730.332654] env[71191]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 730.332654] env[71191]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 730.332654] env[71191]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 730.332654] env[71191]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 730.332654] env[71191]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 730.332654] env[71191]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 730.332654] env[71191]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 730.332654] env[71191]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 730.332654] env[71191]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 730.332654] env[71191]: ERROR neutron response = self.get(path=path, params=params) [ 730.333406] env[71191]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 730.333406] env[71191]: ERROR neutron raise RuntimeError(msg, last_err) [ 730.333406] env[71191]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 730.333406] env[71191]: ERROR neutron [ 733.070404] env[71313]: INFO neutron.common.config [-] Logging enabled! [ 733.070741] env[71313]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 733.070952] env[71313]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71313) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 733.071181] env[71313]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 733.071265] env[71313]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 733.072146] env[71313]: WARNING oslo_policy.policy [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 733.072368] env[71313]: WARNING oslo_policy.policy [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 733.073587] env[71313]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 733.088705] env[71313]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Session token - acquiring [ 733.089587] env[71313]: DEBUG networking_nsxv3.common.locking [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71313) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 733.089836] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 733.090004] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 733.090414] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 733.100144] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 733.100462] env[71313]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 733.100638] env[71313]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71313) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 735.106704] env[71313]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Session token - acquiring [ 735.107085] env[71313]: DEBUG networking_nsxv3.common.locking [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71313) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 735.107277] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 735.107416] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 735.107691] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 735.116938] env[71313]: DEBUG oslo_concurrency.lockutils [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71313) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 735.117182] env[71313]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 735.117318] env[71313]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71313) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 737.120899] env[71313]: CRITICAL neutron [None req-b54173ed-3fa2-4e0d-87d9-8624c3c8d981 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 737.120899] env[71313]: ERROR neutron Traceback (most recent call last): [ 737.120899] env[71313]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 737.120899] env[71313]: ERROR neutron sys.exit(main()) [ 737.120899] env[71313]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 737.120899] env[71313]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 737.120899] env[71313]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 737.120899] env[71313]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 737.120899] env[71313]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 737.120899] env[71313]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 737.120899] env[71313]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 737.120899] env[71313]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 737.120899] env[71313]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 737.120899] env[71313]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 737.120899] env[71313]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 737.120899] env[71313]: ERROR neutron response = self.get(path=path, params=params) [ 737.121573] env[71313]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 737.121573] env[71313]: ERROR neutron raise RuntimeError(msg, last_err) [ 737.121573] env[71313]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 737.121573] env[71313]: ERROR neutron [ 739.793753] env[71484]: INFO neutron.common.config [-] Logging enabled! [ 739.794117] env[71484]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 739.797773] env[71484]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71484) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 739.797773] env[71484]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 739.797773] env[71484]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 739.797773] env[71484]: WARNING oslo_policy.policy [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 739.797773] env[71484]: WARNING oslo_policy.policy [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 739.797773] env[71484]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 739.811589] env[71484]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Session token - acquiring [ 739.812545] env[71484]: DEBUG networking_nsxv3.common.locking [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71484) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 739.812764] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 739.813657] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 739.814073] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 739.823365] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 739.823626] env[71484]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 739.823774] env[71484]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71484) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 741.829856] env[71484]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Session token - acquiring [ 741.830343] env[71484]: DEBUG networking_nsxv3.common.locking [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71484) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 741.830559] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 741.830689] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 741.830967] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 741.839946] env[71484]: DEBUG oslo_concurrency.lockutils [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71484) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 741.840211] env[71484]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 741.840353] env[71484]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71484) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 743.843253] env[71484]: CRITICAL neutron [None req-ca260f62-d175-4348-bfad-7b69b66f2ec2 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 743.843253] env[71484]: ERROR neutron Traceback (most recent call last): [ 743.843253] env[71484]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 743.843253] env[71484]: ERROR neutron sys.exit(main()) [ 743.843253] env[71484]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 743.843253] env[71484]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 743.843253] env[71484]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 743.843253] env[71484]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 743.843253] env[71484]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 743.843253] env[71484]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 743.843253] env[71484]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 743.843253] env[71484]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 743.843253] env[71484]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 743.843253] env[71484]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 743.843253] env[71484]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 743.843253] env[71484]: ERROR neutron response = self.get(path=path, params=params) [ 743.844093] env[71484]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 743.844093] env[71484]: ERROR neutron raise RuntimeError(msg, last_err) [ 743.844093] env[71484]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 743.844093] env[71484]: ERROR neutron [ 746.501247] env[71601]: INFO neutron.common.config [-] Logging enabled! [ 746.501592] env[71601]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 746.501769] env[71601]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71601) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 746.501985] env[71601]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 746.502096] env[71601]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 746.502912] env[71601]: WARNING oslo_policy.policy [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 746.503182] env[71601]: WARNING oslo_policy.policy [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 746.504472] env[71601]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 746.521591] env[71601]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Session token - acquiring [ 746.522402] env[71601]: DEBUG networking_nsxv3.common.locking [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71601) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 746.522607] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 746.522732] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 746.523094] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 746.533120] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 746.533412] env[71601]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 746.533587] env[71601]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71601) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 748.539739] env[71601]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Session token - acquiring [ 748.540173] env[71601]: DEBUG networking_nsxv3.common.locking [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71601) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 748.540382] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 748.540510] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 748.540788] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 748.550026] env[71601]: DEBUG oslo_concurrency.lockutils [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71601) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 748.550264] env[71601]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 748.550425] env[71601]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71601) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 750.553344] env[71601]: CRITICAL neutron [None req-d1fb06b0-3b53-49fc-b283-74f46b16e8ec None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 750.553344] env[71601]: ERROR neutron Traceback (most recent call last): [ 750.553344] env[71601]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 750.553344] env[71601]: ERROR neutron sys.exit(main()) [ 750.553344] env[71601]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 750.553344] env[71601]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 750.553344] env[71601]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 750.553344] env[71601]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 750.553344] env[71601]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 750.553344] env[71601]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 750.553344] env[71601]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 750.553344] env[71601]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 750.553344] env[71601]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 750.553344] env[71601]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 750.553344] env[71601]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 750.553344] env[71601]: ERROR neutron response = self.get(path=path, params=params) [ 750.554262] env[71601]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 750.554262] env[71601]: ERROR neutron raise RuntimeError(msg, last_err) [ 750.554262] env[71601]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 750.554262] env[71601]: ERROR neutron [ 752.965841] env[71716]: INFO neutron.common.config [-] Logging enabled! [ 752.966194] env[71716]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 752.966399] env[71716]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71716) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 752.966629] env[71716]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 752.966723] env[71716]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 752.967604] env[71716]: WARNING oslo_policy.policy [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 752.967823] env[71716]: WARNING oslo_policy.policy [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 752.969086] env[71716]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 752.983173] env[71716]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Session token - acquiring [ 752.983998] env[71716]: DEBUG networking_nsxv3.common.locking [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71716) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 752.984309] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 752.984348] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 752.984712] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 752.993883] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 752.994138] env[71716]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 752.994286] env[71716]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71716) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 755.001796] env[71716]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Session token - acquiring [ 755.002108] env[71716]: DEBUG networking_nsxv3.common.locking [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71716) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 755.002267] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 755.002386] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 755.002663] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 755.012338] env[71716]: DEBUG oslo_concurrency.lockutils [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71716) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 755.012338] env[71716]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 755.012338] env[71716]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71716) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 757.015436] env[71716]: CRITICAL neutron [None req-e5c05e79-9faf-45ec-899a-5cb63cf0dcac None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 757.015436] env[71716]: ERROR neutron Traceback (most recent call last): [ 757.015436] env[71716]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 757.015436] env[71716]: ERROR neutron sys.exit(main()) [ 757.015436] env[71716]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 757.015436] env[71716]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 757.015436] env[71716]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 757.015436] env[71716]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 757.015436] env[71716]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 757.015436] env[71716]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 757.015436] env[71716]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 757.015436] env[71716]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 757.015436] env[71716]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 757.015436] env[71716]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 757.015436] env[71716]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 757.015436] env[71716]: ERROR neutron response = self.get(path=path, params=params) [ 757.016384] env[71716]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 757.016384] env[71716]: ERROR neutron raise RuntimeError(msg, last_err) [ 757.016384] env[71716]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 757.016384] env[71716]: ERROR neutron [ 759.883580] env[71821]: INFO neutron.common.config [-] Logging enabled! [ 759.883580] env[71821]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 759.884220] env[71821]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71821) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 759.884433] env[71821]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 759.884528] env[71821]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 759.885744] env[71821]: WARNING oslo_policy.policy [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 759.885998] env[71821]: WARNING oslo_policy.policy [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 759.887274] env[71821]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 759.904773] env[71821]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Session token - acquiring [ 759.904773] env[71821]: DEBUG networking_nsxv3.common.locking [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71821) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 759.904773] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 759.906104] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 759.906104] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 759.931438] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 759.931438] env[71821]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 759.931438] env[71821]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71821) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 761.937541] env[71821]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Session token - acquiring [ 761.938043] env[71821]: DEBUG networking_nsxv3.common.locking [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71821) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 761.938281] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 761.938420] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 761.939683] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 761.951765] env[71821]: DEBUG oslo_concurrency.lockutils [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71821) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 761.952094] env[71821]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 761.952266] env[71821]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71821) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 763.955120] env[71821]: CRITICAL neutron [None req-14d3253b-7707-40bb-b132-976f12e40783 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 763.955120] env[71821]: ERROR neutron Traceback (most recent call last): [ 763.955120] env[71821]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 763.955120] env[71821]: ERROR neutron sys.exit(main()) [ 763.955120] env[71821]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 763.955120] env[71821]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 763.955120] env[71821]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 763.955120] env[71821]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 763.955120] env[71821]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 763.955120] env[71821]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 763.955120] env[71821]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 763.955120] env[71821]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 763.955120] env[71821]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 763.955120] env[71821]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 763.955120] env[71821]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 763.955120] env[71821]: ERROR neutron response = self.get(path=path, params=params) [ 763.955662] env[71821]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 763.955662] env[71821]: ERROR neutron raise RuntimeError(msg, last_err) [ 763.955662] env[71821]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 763.955662] env[71821]: ERROR neutron [ 766.786351] env[72174]: INFO neutron.common.config [-] Logging enabled! [ 766.786643] env[72174]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 766.786888] env[72174]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72174) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 766.787123] env[72174]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 766.787215] env[72174]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 766.788068] env[72174]: WARNING oslo_policy.policy [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 766.788307] env[72174]: WARNING oslo_policy.policy [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 766.789545] env[72174]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 766.814466] env[72174]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Session token - acquiring [ 766.815329] env[72174]: DEBUG networking_nsxv3.common.locking [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72174) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 766.815540] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 766.815662] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 766.816099] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 766.825430] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 766.826415] env[72174]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 766.826415] env[72174]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72174) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 768.833866] env[72174]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Session token - acquiring [ 768.834240] env[72174]: DEBUG networking_nsxv3.common.locking [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72174) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 768.834352] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 768.834478] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 768.834766] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 768.843728] env[72174]: DEBUG oslo_concurrency.lockutils [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72174) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 768.843975] env[72174]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 768.844179] env[72174]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72174) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 770.847624] env[72174]: CRITICAL neutron [None req-c14bc5f6-b586-4656-81f7-5efb7c7bef13 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 770.847624] env[72174]: ERROR neutron Traceback (most recent call last): [ 770.847624] env[72174]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 770.847624] env[72174]: ERROR neutron sys.exit(main()) [ 770.847624] env[72174]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 770.847624] env[72174]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 770.847624] env[72174]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 770.847624] env[72174]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 770.847624] env[72174]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 770.847624] env[72174]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 770.847624] env[72174]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 770.847624] env[72174]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 770.847624] env[72174]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 770.847624] env[72174]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 770.847624] env[72174]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 770.847624] env[72174]: ERROR neutron response = self.get(path=path, params=params) [ 770.848333] env[72174]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 770.848333] env[72174]: ERROR neutron raise RuntimeError(msg, last_err) [ 770.848333] env[72174]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 770.848333] env[72174]: ERROR neutron [ 773.573826] env[72306]: INFO neutron.common.config [-] Logging enabled! [ 773.574258] env[72306]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 773.574399] env[72306]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72306) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 773.574637] env[72306]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 773.574727] env[72306]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 773.575621] env[72306]: WARNING oslo_policy.policy [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 773.575842] env[72306]: WARNING oslo_policy.policy [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 773.577173] env[72306]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 773.592540] env[72306]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Session token - acquiring [ 773.593437] env[72306]: DEBUG networking_nsxv3.common.locking [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72306) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 773.593645] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 773.593769] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 773.594158] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 773.603961] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 773.604344] env[72306]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 773.604509] env[72306]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72306) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 775.610302] env[72306]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Session token - acquiring [ 775.610666] env[72306]: DEBUG networking_nsxv3.common.locking [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72306) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 775.610799] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 775.610924] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 775.611207] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 775.620605] env[72306]: DEBUG oslo_concurrency.lockutils [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72306) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 775.620919] env[72306]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 775.621109] env[72306]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72306) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 777.622785] env[72306]: CRITICAL neutron [None req-ab656acd-51b3-4df2-bcc6-d5323544a725 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 777.622785] env[72306]: ERROR neutron Traceback (most recent call last): [ 777.622785] env[72306]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 777.622785] env[72306]: ERROR neutron sys.exit(main()) [ 777.622785] env[72306]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 777.622785] env[72306]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 777.622785] env[72306]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 777.622785] env[72306]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 777.622785] env[72306]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 777.622785] env[72306]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 777.622785] env[72306]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 777.622785] env[72306]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 777.622785] env[72306]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 777.622785] env[72306]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 777.622785] env[72306]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 777.622785] env[72306]: ERROR neutron response = self.get(path=path, params=params) [ 777.623499] env[72306]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 777.623499] env[72306]: ERROR neutron raise RuntimeError(msg, last_err) [ 777.623499] env[72306]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 777.623499] env[72306]: ERROR neutron [ 780.275375] env[72431]: INFO neutron.common.config [-] Logging enabled! [ 780.275692] env[72431]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 780.275908] env[72431]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72431) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 780.276152] env[72431]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 780.276246] env[72431]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 780.277139] env[72431]: WARNING oslo_policy.policy [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 780.277342] env[72431]: WARNING oslo_policy.policy [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 780.278574] env[72431]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 780.298151] env[72431]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Session token - acquiring [ 780.299309] env[72431]: DEBUG networking_nsxv3.common.locking [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72431) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 780.299549] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 780.299637] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 780.300058] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 780.310305] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 780.310566] env[72431]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 780.310732] env[72431]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72431) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 782.316851] env[72431]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Session token - acquiring [ 782.317245] env[72431]: DEBUG networking_nsxv3.common.locking [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72431) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 782.317452] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 782.317556] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 782.317841] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 782.327131] env[72431]: DEBUG oslo_concurrency.lockutils [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72431) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 782.327472] env[72431]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 782.327641] env[72431]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72431) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 784.328920] env[72431]: CRITICAL neutron [None req-328992a6-e2b9-4e32-ba32-9a4a9d4e8c2f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 784.328920] env[72431]: ERROR neutron Traceback (most recent call last): [ 784.328920] env[72431]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 784.328920] env[72431]: ERROR neutron sys.exit(main()) [ 784.328920] env[72431]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 784.328920] env[72431]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 784.328920] env[72431]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 784.328920] env[72431]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 784.328920] env[72431]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 784.328920] env[72431]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 784.328920] env[72431]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 784.328920] env[72431]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 784.328920] env[72431]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 784.328920] env[72431]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 784.328920] env[72431]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 784.328920] env[72431]: ERROR neutron response = self.get(path=path, params=params) [ 784.329648] env[72431]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 784.329648] env[72431]: ERROR neutron raise RuntimeError(msg, last_err) [ 784.329648] env[72431]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 784.329648] env[72431]: ERROR neutron [ 786.994638] env[72548]: INFO neutron.common.config [-] Logging enabled! [ 786.994984] env[72548]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 786.995165] env[72548]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72548) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 786.995401] env[72548]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 786.995496] env[72548]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 786.996428] env[72548]: WARNING oslo_policy.policy [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 786.996657] env[72548]: WARNING oslo_policy.policy [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 786.997884] env[72548]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 787.012844] env[72548]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Session token - acquiring [ 787.013660] env[72548]: DEBUG networking_nsxv3.common.locking [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72548) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 787.013865] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 787.014066] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 787.014502] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 787.024531] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 787.024798] env[72548]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 787.024949] env[72548]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72548) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 789.031274] env[72548]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Session token - acquiring [ 789.032146] env[72548]: DEBUG networking_nsxv3.common.locking [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72548) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 789.032146] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 789.032297] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 789.032297] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 789.041768] env[72548]: DEBUG oslo_concurrency.lockutils [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72548) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 789.042095] env[72548]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 789.042278] env[72548]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72548) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 791.044896] env[72548]: CRITICAL neutron [None req-f1f74c7a-1c41-411a-946f-411a2c16b885 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 791.044896] env[72548]: ERROR neutron Traceback (most recent call last): [ 791.044896] env[72548]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 791.044896] env[72548]: ERROR neutron sys.exit(main()) [ 791.044896] env[72548]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 791.044896] env[72548]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 791.044896] env[72548]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 791.044896] env[72548]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 791.044896] env[72548]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 791.044896] env[72548]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 791.044896] env[72548]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 791.044896] env[72548]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 791.044896] env[72548]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 791.044896] env[72548]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 791.044896] env[72548]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 791.044896] env[72548]: ERROR neutron response = self.get(path=path, params=params) [ 791.045855] env[72548]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 791.045855] env[72548]: ERROR neutron raise RuntimeError(msg, last_err) [ 791.045855] env[72548]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 791.045855] env[72548]: ERROR neutron [ 793.491873] env[72641]: INFO neutron.common.config [-] Logging enabled! [ 793.492170] env[72641]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 793.492424] env[72641]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72641) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 793.492653] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 793.492744] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 793.493588] env[72641]: WARNING oslo_policy.policy [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 793.493817] env[72641]: WARNING oslo_policy.policy [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 793.495026] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 793.509321] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Session token - acquiring [ 793.510071] env[72641]: DEBUG networking_nsxv3.common.locking [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72641) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 793.510271] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 793.510401] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 793.510762] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 793.519558] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 793.519803] env[72641]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 793.519950] env[72641]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72641) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 795.525865] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Session token - acquiring [ 795.526376] env[72641]: DEBUG networking_nsxv3.common.locking [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72641) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 795.526566] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 795.526697] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 795.527024] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 795.536345] env[72641]: DEBUG oslo_concurrency.lockutils [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 795.536582] env[72641]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 795.536719] env[72641]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72641) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 797.539646] env[72641]: CRITICAL neutron [None req-abab2f1c-2781-4e3c-8283-175dd5646d2f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 797.539646] env[72641]: ERROR neutron Traceback (most recent call last): [ 797.539646] env[72641]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 797.539646] env[72641]: ERROR neutron sys.exit(main()) [ 797.539646] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 797.539646] env[72641]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 797.539646] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 797.539646] env[72641]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 797.539646] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 797.539646] env[72641]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 797.539646] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 797.539646] env[72641]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 797.539646] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 797.539646] env[72641]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 797.539646] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 797.539646] env[72641]: ERROR neutron response = self.get(path=path, params=params) [ 797.540278] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 797.540278] env[72641]: ERROR neutron raise RuntimeError(msg, last_err) [ 797.540278] env[72641]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 797.540278] env[72641]: ERROR neutron [ 799.959696] env[72711]: INFO neutron.common.config [-] Logging enabled! [ 799.960044] env[72711]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 799.960252] env[72711]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72711) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 799.960480] env[72711]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 799.960570] env[72711]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 799.961494] env[72711]: WARNING oslo_policy.policy [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 799.961637] env[72711]: WARNING oslo_policy.policy [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 799.962842] env[72711]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 799.976260] env[72711]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Session token - acquiring [ 799.977377] env[72711]: DEBUG networking_nsxv3.common.locking [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72711) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 799.977377] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 799.977579] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 799.977743] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 799.988599] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 799.988876] env[72711]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 799.989028] env[72711]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72711) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 801.997069] env[72711]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Session token - acquiring [ 801.997410] env[72711]: DEBUG networking_nsxv3.common.locking [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72711) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 801.997597] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 801.997725] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 801.998007] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 802.007042] env[72711]: DEBUG oslo_concurrency.lockutils [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72711) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 802.007298] env[72711]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 802.007439] env[72711]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72711) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 804.008914] env[72711]: CRITICAL neutron [None req-f7631ad2-6661-4be4-8576-171ebdf4b35e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 804.008914] env[72711]: ERROR neutron Traceback (most recent call last): [ 804.008914] env[72711]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 804.008914] env[72711]: ERROR neutron sys.exit(main()) [ 804.008914] env[72711]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 804.008914] env[72711]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 804.008914] env[72711]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 804.008914] env[72711]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 804.008914] env[72711]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 804.008914] env[72711]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 804.008914] env[72711]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 804.008914] env[72711]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 804.008914] env[72711]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 804.008914] env[72711]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 804.008914] env[72711]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 804.008914] env[72711]: ERROR neutron response = self.get(path=path, params=params) [ 804.009517] env[72711]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 804.009517] env[72711]: ERROR neutron raise RuntimeError(msg, last_err) [ 804.009517] env[72711]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 804.009517] env[72711]: ERROR neutron [ 806.750567] env[72729]: INFO neutron.common.config [-] Logging enabled! [ 806.750916] env[72729]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 806.751099] env[72729]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72729) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 806.751298] env[72729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 806.751388] env[72729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 806.752227] env[72729]: WARNING oslo_policy.policy [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 806.752450] env[72729]: WARNING oslo_policy.policy [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 806.753631] env[72729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 806.772187] env[72729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Session token - acquiring [ 806.772991] env[72729]: DEBUG networking_nsxv3.common.locking [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72729) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 806.773199] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 806.773340] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 806.773710] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 806.783162] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 806.783410] env[72729]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 806.783555] env[72729]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72729) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 808.789830] env[72729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Session token - acquiring [ 808.790302] env[72729]: DEBUG networking_nsxv3.common.locking [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72729) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 808.790607] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 808.790828] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 808.791261] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 808.802745] env[72729]: DEBUG oslo_concurrency.lockutils [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72729) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 808.803087] env[72729]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 808.803293] env[72729]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72729) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 810.805390] env[72729]: CRITICAL neutron [None req-9c58f3fb-ab72-47b3-87a5-f926a2707619 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 810.805390] env[72729]: ERROR neutron Traceback (most recent call last): [ 810.805390] env[72729]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 810.805390] env[72729]: ERROR neutron sys.exit(main()) [ 810.805390] env[72729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 810.805390] env[72729]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 810.805390] env[72729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 810.805390] env[72729]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 810.805390] env[72729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 810.805390] env[72729]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 810.805390] env[72729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 810.805390] env[72729]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 810.805390] env[72729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 810.805390] env[72729]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 810.805390] env[72729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 810.805390] env[72729]: ERROR neutron response = self.get(path=path, params=params) [ 810.806340] env[72729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 810.806340] env[72729]: ERROR neutron raise RuntimeError(msg, last_err) [ 810.806340] env[72729]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 810.806340] env[72729]: ERROR neutron [ 813.514499] env[72799]: INFO neutron.common.config [-] Logging enabled! [ 813.514828] env[72799]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 813.515040] env[72799]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72799) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 813.515198] env[72799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 813.515364] env[72799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 813.516276] env[72799]: WARNING oslo_policy.policy [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 813.516500] env[72799]: WARNING oslo_policy.policy [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 813.517786] env[72799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 813.538680] env[72799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Session token - acquiring [ 813.539557] env[72799]: DEBUG networking_nsxv3.common.locking [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72799) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 813.540106] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 813.540106] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 813.540318] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 813.554231] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 813.554308] env[72799]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 813.554396] env[72799]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72799) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 815.562339] env[72799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Session token - acquiring [ 815.562743] env[72799]: DEBUG networking_nsxv3.common.locking [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72799) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 815.562743] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 815.562867] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 815.563076] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 815.572996] env[72799]: DEBUG oslo_concurrency.lockutils [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 815.573245] env[72799]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 815.573383] env[72799]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72799) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 817.575112] env[72799]: CRITICAL neutron [None req-caf053c9-186f-4ba6-bb72-e0e47869b37d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 817.575112] env[72799]: ERROR neutron Traceback (most recent call last): [ 817.575112] env[72799]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 817.575112] env[72799]: ERROR neutron sys.exit(main()) [ 817.575112] env[72799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 817.575112] env[72799]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 817.575112] env[72799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 817.575112] env[72799]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 817.575112] env[72799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 817.575112] env[72799]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 817.575112] env[72799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 817.575112] env[72799]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 817.575112] env[72799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 817.575112] env[72799]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 817.575112] env[72799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 817.575112] env[72799]: ERROR neutron response = self.get(path=path, params=params) [ 817.575852] env[72799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 817.575852] env[72799]: ERROR neutron raise RuntimeError(msg, last_err) [ 817.575852] env[72799]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 817.575852] env[72799]: ERROR neutron [ 820.271977] env[72901]: INFO neutron.common.config [-] Logging enabled! [ 820.271977] env[72901]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 820.272392] env[72901]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72901) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 820.272461] env[72901]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 820.272556] env[72901]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 820.273449] env[72901]: WARNING oslo_policy.policy [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 820.273670] env[72901]: WARNING oslo_policy.policy [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 820.274926] env[72901]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 820.289552] env[72901]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Session token - acquiring [ 820.290364] env[72901]: DEBUG networking_nsxv3.common.locking [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72901) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 820.290578] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 820.290706] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 820.291074] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 820.300191] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 820.300446] env[72901]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 820.300591] env[72901]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72901) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 822.306562] env[72901]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Session token - acquiring [ 822.306884] env[72901]: DEBUG networking_nsxv3.common.locking [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72901) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 822.307074] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 822.307284] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 822.307480] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 822.317163] env[72901]: DEBUG oslo_concurrency.lockutils [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72901) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 822.317403] env[72901]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 822.317539] env[72901]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72901) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 824.320154] env[72901]: CRITICAL neutron [None req-b1026678-899a-40ba-ac51-4ed45b10c3a8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 824.320154] env[72901]: ERROR neutron Traceback (most recent call last): [ 824.320154] env[72901]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 824.320154] env[72901]: ERROR neutron sys.exit(main()) [ 824.320154] env[72901]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 824.320154] env[72901]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 824.320154] env[72901]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 824.320154] env[72901]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 824.320154] env[72901]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 824.320154] env[72901]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 824.320154] env[72901]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 824.320154] env[72901]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 824.320154] env[72901]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 824.320154] env[72901]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 824.320154] env[72901]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 824.320154] env[72901]: ERROR neutron response = self.get(path=path, params=params) [ 824.321081] env[72901]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 824.321081] env[72901]: ERROR neutron raise RuntimeError(msg, last_err) [ 824.321081] env[72901]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 824.321081] env[72901]: ERROR neutron [ 827.074323] env[72933]: INFO neutron.common.config [-] Logging enabled! [ 827.074647] env[72933]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 827.074858] env[72933]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72933) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 827.075091] env[72933]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 827.075180] env[72933]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 827.076055] env[72933]: WARNING oslo_policy.policy [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 827.076276] env[72933]: WARNING oslo_policy.policy [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 827.077504] env[72933]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 827.091690] env[72933]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Session token - acquiring [ 827.092526] env[72933]: DEBUG networking_nsxv3.common.locking [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72933) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 827.096974] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 827.096974] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 827.096974] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 827.106079] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 827.106415] env[72933]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 827.106642] env[72933]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72933) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 829.114889] env[72933]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Session token - acquiring [ 829.114889] env[72933]: DEBUG networking_nsxv3.common.locking [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72933) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 829.114889] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 829.115207] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 829.115207] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 829.123191] env[72933]: DEBUG oslo_concurrency.lockutils [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72933) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 829.123582] env[72933]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 829.123817] env[72933]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72933) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 831.124866] env[72933]: CRITICAL neutron [None req-b244f14b-ab59-4a61-a4f0-2293cf895a47 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 831.124866] env[72933]: ERROR neutron Traceback (most recent call last): [ 831.124866] env[72933]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 831.124866] env[72933]: ERROR neutron sys.exit(main()) [ 831.124866] env[72933]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 831.124866] env[72933]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 831.124866] env[72933]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 831.124866] env[72933]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 831.124866] env[72933]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 831.124866] env[72933]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 831.124866] env[72933]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 831.124866] env[72933]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 831.124866] env[72933]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 831.124866] env[72933]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 831.124866] env[72933]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 831.124866] env[72933]: ERROR neutron response = self.get(path=path, params=params) [ 831.125657] env[72933]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 831.125657] env[72933]: ERROR neutron raise RuntimeError(msg, last_err) [ 831.125657] env[72933]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 831.125657] env[72933]: ERROR neutron [ 833.753882] env[73003]: INFO neutron.common.config [-] Logging enabled! [ 833.754176] env[73003]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 833.754410] env[73003]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73003) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 833.754640] env[73003]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 833.754735] env[73003]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 833.755575] env[73003]: WARNING oslo_policy.policy [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 833.755796] env[73003]: WARNING oslo_policy.policy [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 833.757043] env[73003]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 833.770855] env[73003]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Session token - acquiring [ 833.771665] env[73003]: DEBUG networking_nsxv3.common.locking [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73003) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 833.771860] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 833.771992] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 833.772820] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 833.782278] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 833.782544] env[73003]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 833.782694] env[73003]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73003) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 835.789911] env[73003]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Session token - acquiring [ 835.790312] env[73003]: DEBUG networking_nsxv3.common.locking [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73003) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 835.790386] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 835.790516] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 835.790790] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 835.800373] env[73003]: DEBUG oslo_concurrency.lockutils [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73003) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 835.800616] env[73003]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 835.800754] env[73003]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73003) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 837.801944] env[73003]: CRITICAL neutron [None req-a3c9e58c-70ea-46bd-8d2a-671b607283f8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 837.801944] env[73003]: ERROR neutron Traceback (most recent call last): [ 837.801944] env[73003]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 837.801944] env[73003]: ERROR neutron sys.exit(main()) [ 837.801944] env[73003]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 837.801944] env[73003]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 837.801944] env[73003]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 837.801944] env[73003]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 837.801944] env[73003]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 837.801944] env[73003]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 837.801944] env[73003]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 837.801944] env[73003]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 837.801944] env[73003]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 837.801944] env[73003]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 837.801944] env[73003]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 837.801944] env[73003]: ERROR neutron response = self.get(path=path, params=params) [ 837.802626] env[73003]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 837.802626] env[73003]: ERROR neutron raise RuntimeError(msg, last_err) [ 837.802626] env[73003]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 837.802626] env[73003]: ERROR neutron [ 840.491801] env[73043]: INFO neutron.common.config [-] Logging enabled! [ 840.492129] env[73043]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 840.492333] env[73043]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73043) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 840.492551] env[73043]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 840.492663] env[73043]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 840.493482] env[73043]: WARNING oslo_policy.policy [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 840.493694] env[73043]: WARNING oslo_policy.policy [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 840.494904] env[73043]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 840.509025] env[73043]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Session token - acquiring [ 840.509801] env[73043]: DEBUG networking_nsxv3.common.locking [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73043) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 840.509996] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 840.510568] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 840.510568] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 840.519178] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 840.519428] env[73043]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 840.519572] env[73043]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73043) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 842.525967] env[73043]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Session token - acquiring [ 842.528416] env[73043]: DEBUG networking_nsxv3.common.locking [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73043) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 842.528416] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 842.528554] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 842.528554] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 842.538603] env[73043]: DEBUG oslo_concurrency.lockutils [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73043) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 842.538987] env[73043]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 842.539257] env[73043]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73043) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 844.540892] env[73043]: CRITICAL neutron [None req-b825b580-5ce3-479c-84ef-49d10aa02076 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 844.540892] env[73043]: ERROR neutron Traceback (most recent call last): [ 844.540892] env[73043]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 844.540892] env[73043]: ERROR neutron sys.exit(main()) [ 844.540892] env[73043]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 844.540892] env[73043]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 844.540892] env[73043]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 844.540892] env[73043]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 844.540892] env[73043]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 844.540892] env[73043]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 844.540892] env[73043]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 844.540892] env[73043]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 844.540892] env[73043]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 844.540892] env[73043]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 844.540892] env[73043]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 844.540892] env[73043]: ERROR neutron response = self.get(path=path, params=params) [ 844.541586] env[73043]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 844.541586] env[73043]: ERROR neutron raise RuntimeError(msg, last_err) [ 844.541586] env[73043]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 844.541586] env[73043]: ERROR neutron [ 847.321388] env[73135]: INFO neutron.common.config [-] Logging enabled! [ 847.321751] env[73135]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 847.321946] env[73135]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73135) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 847.322184] env[73135]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 847.322273] env[73135]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 847.323112] env[73135]: WARNING oslo_policy.policy [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 847.323340] env[73135]: WARNING oslo_policy.policy [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 847.324588] env[73135]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 847.341283] env[73135]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Session token - acquiring [ 847.342388] env[73135]: DEBUG networking_nsxv3.common.locking [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73135) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 847.343569] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 847.343793] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 847.344388] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 847.354031] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 847.354387] env[73135]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 847.354484] env[73135]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73135) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 849.367810] env[73135]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Session token - acquiring [ 849.367810] env[73135]: DEBUG networking_nsxv3.common.locking [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73135) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 849.367810] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 849.368369] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 849.368369] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 849.372119] env[73135]: DEBUG oslo_concurrency.lockutils [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73135) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 849.372483] env[73135]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 849.372658] env[73135]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73135) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 851.375566] env[73135]: CRITICAL neutron [None req-4ae28f66-2405-4248-af0b-4aa07c988fcc None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 851.375566] env[73135]: ERROR neutron Traceback (most recent call last): [ 851.375566] env[73135]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 851.375566] env[73135]: ERROR neutron sys.exit(main()) [ 851.375566] env[73135]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 851.375566] env[73135]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 851.375566] env[73135]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 851.375566] env[73135]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 851.375566] env[73135]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 851.375566] env[73135]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 851.375566] env[73135]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 851.375566] env[73135]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 851.375566] env[73135]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 851.375566] env[73135]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 851.375566] env[73135]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 851.375566] env[73135]: ERROR neutron response = self.get(path=path, params=params) [ 851.376402] env[73135]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 851.376402] env[73135]: ERROR neutron raise RuntimeError(msg, last_err) [ 851.376402] env[73135]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 851.376402] env[73135]: ERROR neutron [ 854.004247] env[73346]: INFO neutron.common.config [-] Logging enabled! [ 854.004582] env[73346]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 854.004773] env[73346]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73346) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 854.005024] env[73346]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 854.005121] env[73346]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 854.005992] env[73346]: WARNING oslo_policy.policy [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 854.006239] env[73346]: WARNING oslo_policy.policy [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 854.007419] env[73346]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 854.022081] env[73346]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Session token - acquiring [ 854.022848] env[73346]: DEBUG networking_nsxv3.common.locking [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73346) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 854.023051] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 854.023181] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 854.023541] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 854.032353] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 854.032567] env[73346]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 854.032710] env[73346]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73346) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 856.038839] env[73346]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Session token - acquiring [ 856.039208] env[73346]: DEBUG networking_nsxv3.common.locking [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73346) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 856.039419] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 856.039546] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 856.039824] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 856.049635] env[73346]: DEBUG oslo_concurrency.lockutils [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73346) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 856.049871] env[73346]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 856.050012] env[73346]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73346) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 858.052954] env[73346]: CRITICAL neutron [None req-156c97c6-6643-4e8f-9bdc-96f2e5f651e5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 858.052954] env[73346]: ERROR neutron Traceback (most recent call last): [ 858.052954] env[73346]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 858.052954] env[73346]: ERROR neutron sys.exit(main()) [ 858.052954] env[73346]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 858.052954] env[73346]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 858.052954] env[73346]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 858.052954] env[73346]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 858.052954] env[73346]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 858.052954] env[73346]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 858.052954] env[73346]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 858.052954] env[73346]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 858.052954] env[73346]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 858.052954] env[73346]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 858.052954] env[73346]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 858.052954] env[73346]: ERROR neutron response = self.get(path=path, params=params) [ 858.053708] env[73346]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 858.053708] env[73346]: ERROR neutron raise RuntimeError(msg, last_err) [ 858.053708] env[73346]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 858.053708] env[73346]: ERROR neutron [ 860.818104] env[73487]: INFO neutron.common.config [-] Logging enabled! [ 860.818425] env[73487]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 860.818637] env[73487]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73487) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 860.818888] env[73487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 860.818973] env[73487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 860.819911] env[73487]: WARNING oslo_policy.policy [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 860.820204] env[73487]: WARNING oslo_policy.policy [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 860.821714] env[73487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 860.837354] env[73487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Session token - acquiring [ 860.838187] env[73487]: DEBUG networking_nsxv3.common.locking [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73487) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 860.838390] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 860.838515] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 860.838894] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 860.848228] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 860.848478] env[73487]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 860.848620] env[73487]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73487) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 862.854806] env[73487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Session token - acquiring [ 862.855109] env[73487]: DEBUG networking_nsxv3.common.locking [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73487) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 862.855308] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 862.855448] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 862.855722] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 862.865605] env[73487]: DEBUG oslo_concurrency.lockutils [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 862.865890] env[73487]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 862.866040] env[73487]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73487) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 864.869088] env[73487]: CRITICAL neutron [None req-d6b14ea1-f4bc-4cbe-8290-76483c7b59e6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 864.869088] env[73487]: ERROR neutron Traceback (most recent call last): [ 864.869088] env[73487]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 864.869088] env[73487]: ERROR neutron sys.exit(main()) [ 864.869088] env[73487]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 864.869088] env[73487]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 864.869088] env[73487]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 864.869088] env[73487]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 864.869088] env[73487]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 864.869088] env[73487]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 864.869088] env[73487]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 864.869088] env[73487]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 864.869088] env[73487]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 864.869088] env[73487]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 864.869088] env[73487]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 864.869088] env[73487]: ERROR neutron response = self.get(path=path, params=params) [ 864.869851] env[73487]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 864.869851] env[73487]: ERROR neutron raise RuntimeError(msg, last_err) [ 864.869851] env[73487]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 864.869851] env[73487]: ERROR neutron [ 867.468330] env[73558]: INFO neutron.common.config [-] Logging enabled! [ 867.468737] env[73558]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 867.469060] env[73558]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73558) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 867.469421] env[73558]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 867.469573] env[73558]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 867.470572] env[73558]: WARNING oslo_policy.policy [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 867.470882] env[73558]: WARNING oslo_policy.policy [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 867.472182] env[73558]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 867.487875] env[73558]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Session token - acquiring [ 867.488783] env[73558]: DEBUG networking_nsxv3.common.locking [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73558) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 867.489015] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 867.489121] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 867.489502] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 867.498851] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 867.499109] env[73558]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 867.499252] env[73558]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73558) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 869.505164] env[73558]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Session token - acquiring [ 869.505602] env[73558]: DEBUG networking_nsxv3.common.locking [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73558) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 869.505800] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 869.505899] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 869.506219] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 869.515201] env[73558]: DEBUG oslo_concurrency.lockutils [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73558) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 869.515453] env[73558]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 869.515595] env[73558]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73558) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 871.516884] env[73558]: CRITICAL neutron [None req-0a414983-c431-49ef-8953-c13954c5c367 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 871.516884] env[73558]: ERROR neutron Traceback (most recent call last): [ 871.516884] env[73558]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 871.516884] env[73558]: ERROR neutron sys.exit(main()) [ 871.516884] env[73558]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 871.516884] env[73558]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 871.516884] env[73558]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 871.516884] env[73558]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 871.516884] env[73558]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 871.516884] env[73558]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 871.516884] env[73558]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 871.516884] env[73558]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 871.516884] env[73558]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 871.516884] env[73558]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 871.516884] env[73558]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 871.516884] env[73558]: ERROR neutron response = self.get(path=path, params=params) [ 871.517659] env[73558]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 871.517659] env[73558]: ERROR neutron raise RuntimeError(msg, last_err) [ 871.517659] env[73558]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 871.517659] env[73558]: ERROR neutron [ 873.946077] env[73587]: INFO neutron.common.config [-] Logging enabled! [ 873.946372] env[73587]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 873.946610] env[73587]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73587) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 873.946831] env[73587]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 873.946924] env[73587]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 873.947802] env[73587]: WARNING oslo_policy.policy [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 873.948102] env[73587]: WARNING oslo_policy.policy [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 873.949470] env[73587]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 873.963952] env[73587]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Session token - acquiring [ 873.964849] env[73587]: DEBUG networking_nsxv3.common.locking [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73587) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 873.965155] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 873.965345] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 873.966175] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 873.975631] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 873.975910] env[73587]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 873.976081] env[73587]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73587) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 875.981943] env[73587]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Session token - acquiring [ 875.982408] env[73587]: DEBUG networking_nsxv3.common.locking [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73587) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 875.982622] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 875.982751] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 875.983027] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 875.992114] env[73587]: DEBUG oslo_concurrency.lockutils [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73587) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 875.992362] env[73587]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 875.992503] env[73587]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73587) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 877.995403] env[73587]: CRITICAL neutron [None req-63bd88e2-f8a7-4dc8-8554-294b1d591ac3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 877.995403] env[73587]: ERROR neutron Traceback (most recent call last): [ 877.995403] env[73587]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 877.995403] env[73587]: ERROR neutron sys.exit(main()) [ 877.995403] env[73587]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 877.995403] env[73587]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 877.995403] env[73587]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 877.995403] env[73587]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 877.995403] env[73587]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 877.995403] env[73587]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 877.995403] env[73587]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 877.995403] env[73587]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 877.995403] env[73587]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 877.995403] env[73587]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 877.995403] env[73587]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 877.995403] env[73587]: ERROR neutron response = self.get(path=path, params=params) [ 877.996138] env[73587]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 877.996138] env[73587]: ERROR neutron raise RuntimeError(msg, last_err) [ 877.996138] env[73587]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 877.996138] env[73587]: ERROR neutron [ 880.833184] env[73686]: INFO neutron.common.config [-] Logging enabled! [ 880.833458] env[73686]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 880.833696] env[73686]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73686) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 880.833952] env[73686]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 880.834050] env[73686]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 880.835135] env[73686]: WARNING oslo_policy.policy [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 880.835387] env[73686]: WARNING oslo_policy.policy [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 880.837242] env[73686]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 880.851817] env[73686]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Session token - acquiring [ 880.853130] env[73686]: DEBUG networking_nsxv3.common.locking [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73686) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 880.853340] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 880.853468] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 880.853843] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 880.863103] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 880.863357] env[73686]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 880.863503] env[73686]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73686) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 882.869536] env[73686]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Session token - acquiring [ 882.869887] env[73686]: DEBUG networking_nsxv3.common.locking [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73686) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 882.870099] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 882.870224] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 882.870523] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 882.880332] env[73686]: DEBUG oslo_concurrency.lockutils [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73686) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 882.880564] env[73686]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 882.880699] env[73686]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73686) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 884.884573] env[73686]: CRITICAL neutron [None req-b4db99a0-f4c6-4963-b613-efef192238f1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 884.884573] env[73686]: ERROR neutron Traceback (most recent call last): [ 884.884573] env[73686]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 884.884573] env[73686]: ERROR neutron sys.exit(main()) [ 884.884573] env[73686]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 884.884573] env[73686]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 884.884573] env[73686]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 884.884573] env[73686]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 884.884573] env[73686]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 884.884573] env[73686]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 884.884573] env[73686]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 884.884573] env[73686]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 884.884573] env[73686]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 884.884573] env[73686]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 884.884573] env[73686]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 884.884573] env[73686]: ERROR neutron response = self.get(path=path, params=params) [ 884.885416] env[73686]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 884.885416] env[73686]: ERROR neutron raise RuntimeError(msg, last_err) [ 884.885416] env[73686]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 884.885416] env[73686]: ERROR neutron [ 887.544610] env[73805]: INFO neutron.common.config [-] Logging enabled! [ 887.544610] env[73805]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 887.544910] env[73805]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73805) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 887.545571] env[73805]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 887.545571] env[73805]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 887.547584] env[73805]: WARNING oslo_policy.policy [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 887.547584] env[73805]: WARNING oslo_policy.policy [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 887.547584] env[73805]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 887.562779] env[73805]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Session token - acquiring [ 887.563639] env[73805]: DEBUG networking_nsxv3.common.locking [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73805) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 887.563799] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 887.564202] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 887.564383] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 887.576360] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 887.576631] env[73805]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 887.576774] env[73805]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73805) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 889.585864] env[73805]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Session token - acquiring [ 889.586349] env[73805]: DEBUG networking_nsxv3.common.locking [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73805) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 889.586424] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 889.586537] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 889.586817] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 889.596479] env[73805]: DEBUG oslo_concurrency.lockutils [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73805) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 889.596730] env[73805]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 889.596867] env[73805]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73805) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 891.600124] env[73805]: CRITICAL neutron [None req-2a641352-b1bc-4b18-b20b-7fa76ea6bf36 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 891.600124] env[73805]: ERROR neutron Traceback (most recent call last): [ 891.600124] env[73805]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 891.600124] env[73805]: ERROR neutron sys.exit(main()) [ 891.600124] env[73805]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 891.600124] env[73805]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 891.600124] env[73805]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 891.600124] env[73805]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 891.600124] env[73805]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 891.600124] env[73805]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 891.600124] env[73805]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 891.600124] env[73805]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 891.600124] env[73805]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 891.600124] env[73805]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 891.600124] env[73805]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 891.600124] env[73805]: ERROR neutron response = self.get(path=path, params=params) [ 891.600735] env[73805]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 891.600735] env[73805]: ERROR neutron raise RuntimeError(msg, last_err) [ 891.600735] env[73805]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 891.600735] env[73805]: ERROR neutron [ 894.277576] env[74161]: INFO neutron.common.config [-] Logging enabled! [ 894.277869] env[74161]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 894.278111] env[74161]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74161) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 894.278334] env[74161]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 894.278425] env[74161]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 894.279265] env[74161]: WARNING oslo_policy.policy [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 894.279475] env[74161]: WARNING oslo_policy.policy [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 894.280684] env[74161]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 894.294988] env[74161]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Session token - acquiring [ 894.295792] env[74161]: DEBUG networking_nsxv3.common.locking [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74161) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 894.296024] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 894.296164] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 894.296619] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 894.310579] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 894.310859] env[74161]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 894.311008] env[74161]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74161) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 896.317496] env[74161]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Session token - acquiring [ 896.318013] env[74161]: DEBUG networking_nsxv3.common.locking [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74161) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 896.318316] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 896.318510] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 896.318883] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 896.329720] env[74161]: DEBUG oslo_concurrency.lockutils [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74161) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 896.330049] env[74161]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 896.330253] env[74161]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74161) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 898.332868] env[74161]: CRITICAL neutron [None req-3841274b-337f-4452-acba-8ae5643dc8da None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 898.332868] env[74161]: ERROR neutron Traceback (most recent call last): [ 898.332868] env[74161]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 898.332868] env[74161]: ERROR neutron sys.exit(main()) [ 898.332868] env[74161]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 898.332868] env[74161]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 898.332868] env[74161]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 898.332868] env[74161]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 898.332868] env[74161]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 898.332868] env[74161]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 898.332868] env[74161]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 898.332868] env[74161]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 898.332868] env[74161]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 898.332868] env[74161]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 898.332868] env[74161]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 898.332868] env[74161]: ERROR neutron response = self.get(path=path, params=params) [ 898.333621] env[74161]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 898.333621] env[74161]: ERROR neutron raise RuntimeError(msg, last_err) [ 898.333621] env[74161]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 898.333621] env[74161]: ERROR neutron [ 901.039114] env[74288]: INFO neutron.common.config [-] Logging enabled! [ 901.039456] env[74288]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 901.039636] env[74288]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74288) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 901.039863] env[74288]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 901.039957] env[74288]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 901.040870] env[74288]: WARNING oslo_policy.policy [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 901.041090] env[74288]: WARNING oslo_policy.policy [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 901.042300] env[74288]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 901.056519] env[74288]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Session token - acquiring [ 901.057363] env[74288]: DEBUG networking_nsxv3.common.locking [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74288) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 901.057561] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 901.057690] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 901.058063] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 901.067842] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 901.068115] env[74288]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 901.068271] env[74288]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74288) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 903.074365] env[74288]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Session token - acquiring [ 903.074716] env[74288]: DEBUG networking_nsxv3.common.locking [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74288) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 903.074909] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 903.075042] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 903.075317] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 903.084711] env[74288]: DEBUG oslo_concurrency.lockutils [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74288) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 903.084947] env[74288]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 903.085091] env[74288]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74288) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 905.088049] env[74288]: CRITICAL neutron [None req-0892fb90-f1f3-4e0b-be02-64e225f4e18c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 905.088049] env[74288]: ERROR neutron Traceback (most recent call last): [ 905.088049] env[74288]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 905.088049] env[74288]: ERROR neutron sys.exit(main()) [ 905.088049] env[74288]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 905.088049] env[74288]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 905.088049] env[74288]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 905.088049] env[74288]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 905.088049] env[74288]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 905.088049] env[74288]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 905.088049] env[74288]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 905.088049] env[74288]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 905.088049] env[74288]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 905.088049] env[74288]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 905.088049] env[74288]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 905.088049] env[74288]: ERROR neutron response = self.get(path=path, params=params) [ 905.088809] env[74288]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 905.088809] env[74288]: ERROR neutron raise RuntimeError(msg, last_err) [ 905.088809] env[74288]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 905.088809] env[74288]: ERROR neutron [ 907.723488] env[74352]: INFO neutron.common.config [-] Logging enabled! [ 907.723488] env[74352]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 907.723488] env[74352]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74352) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 907.723488] env[74352]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 907.723970] env[74352]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 907.724358] env[74352]: WARNING oslo_policy.policy [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 907.724571] env[74352]: WARNING oslo_policy.policy [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 907.725781] env[74352]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 907.740368] env[74352]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Session token - acquiring [ 907.741170] env[74352]: DEBUG networking_nsxv3.common.locking [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74352) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 907.741372] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 907.741494] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 907.741865] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 907.751104] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 907.751374] env[74352]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 907.751513] env[74352]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74352) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 909.758212] env[74352]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Session token - acquiring [ 909.758517] env[74352]: DEBUG networking_nsxv3.common.locking [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74352) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 909.758517] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 909.758649] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 909.758934] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 909.769696] env[74352]: DEBUG oslo_concurrency.lockutils [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74352) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 909.770109] env[74352]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 909.770347] env[74352]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74352) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 911.772949] env[74352]: CRITICAL neutron [None req-8797cca1-1ced-4416-8ae1-9e21b9314e30 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 911.772949] env[74352]: ERROR neutron Traceback (most recent call last): [ 911.772949] env[74352]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 911.772949] env[74352]: ERROR neutron sys.exit(main()) [ 911.772949] env[74352]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 911.772949] env[74352]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 911.772949] env[74352]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 911.772949] env[74352]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 911.772949] env[74352]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 911.772949] env[74352]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 911.772949] env[74352]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 911.772949] env[74352]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 911.772949] env[74352]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 911.772949] env[74352]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 911.772949] env[74352]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 911.772949] env[74352]: ERROR neutron response = self.get(path=path, params=params) [ 911.773693] env[74352]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 911.773693] env[74352]: ERROR neutron raise RuntimeError(msg, last_err) [ 911.773693] env[74352]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 911.773693] env[74352]: ERROR neutron [ 914.529397] env[74447]: INFO neutron.common.config [-] Logging enabled! [ 914.529747] env[74447]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 914.529917] env[74447]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74447) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 914.530151] env[74447]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 914.530248] env[74447]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 914.531091] env[74447]: WARNING oslo_policy.policy [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 914.531316] env[74447]: WARNING oslo_policy.policy [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 914.532542] env[74447]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 914.547617] env[74447]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Session token - acquiring [ 914.548487] env[74447]: DEBUG networking_nsxv3.common.locking [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74447) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 914.548704] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 914.548832] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 914.549219] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 914.558492] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 914.558760] env[74447]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 914.558909] env[74447]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74447) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 916.565239] env[74447]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Session token - acquiring [ 916.565628] env[74447]: DEBUG networking_nsxv3.common.locking [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74447) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 916.565777] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 916.565895] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 916.566210] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 916.576557] env[74447]: DEBUG oslo_concurrency.lockutils [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74447) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 916.576806] env[74447]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 916.576980] env[74447]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74447) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 918.579884] env[74447]: CRITICAL neutron [None req-432f09b9-056b-406c-93e0-3dc29135af87 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 918.579884] env[74447]: ERROR neutron Traceback (most recent call last): [ 918.579884] env[74447]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 918.579884] env[74447]: ERROR neutron sys.exit(main()) [ 918.579884] env[74447]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 918.579884] env[74447]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 918.579884] env[74447]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 918.579884] env[74447]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 918.579884] env[74447]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 918.579884] env[74447]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 918.579884] env[74447]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 918.579884] env[74447]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 918.579884] env[74447]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 918.579884] env[74447]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 918.579884] env[74447]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 918.579884] env[74447]: ERROR neutron response = self.get(path=path, params=params) [ 918.580557] env[74447]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 918.580557] env[74447]: ERROR neutron raise RuntimeError(msg, last_err) [ 918.580557] env[74447]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 918.580557] env[74447]: ERROR neutron [ 921.422390] env[74734]: INFO neutron.common.config [-] Logging enabled! [ 921.422682] env[74734]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 921.423220] env[74734]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74734) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 921.424149] env[74734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 921.424252] env[74734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 921.425116] env[74734]: WARNING oslo_policy.policy [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 921.425342] env[74734]: WARNING oslo_policy.policy [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 921.426658] env[74734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 921.450298] env[74734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Session token - acquiring [ 921.451180] env[74734]: DEBUG networking_nsxv3.common.locking [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74734) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 921.451683] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 921.451801] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 921.452243] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 921.462797] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 921.463175] env[74734]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 921.463331] env[74734]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74734) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 923.469507] env[74734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Session token - acquiring [ 923.469941] env[74734]: DEBUG networking_nsxv3.common.locking [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74734) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 923.469941] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 923.470042] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 923.470335] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 923.484137] env[74734]: DEBUG oslo_concurrency.lockutils [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74734) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 923.484137] env[74734]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 923.484251] env[74734]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74734) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 925.487240] env[74734]: CRITICAL neutron [None req-98b33f81-97ef-49be-b73a-a655732c718e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 925.487240] env[74734]: ERROR neutron Traceback (most recent call last): [ 925.487240] env[74734]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 925.487240] env[74734]: ERROR neutron sys.exit(main()) [ 925.487240] env[74734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 925.487240] env[74734]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 925.487240] env[74734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 925.487240] env[74734]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 925.487240] env[74734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 925.487240] env[74734]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 925.487240] env[74734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 925.487240] env[74734]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 925.487240] env[74734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 925.487240] env[74734]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 925.487240] env[74734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 925.487240] env[74734]: ERROR neutron response = self.get(path=path, params=params) [ 925.488123] env[74734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 925.488123] env[74734]: ERROR neutron raise RuntimeError(msg, last_err) [ 925.488123] env[74734]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 925.488123] env[74734]: ERROR neutron [ 928.291970] env[75190]: INFO neutron.common.config [-] Logging enabled! [ 928.292245] env[75190]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 928.292656] env[75190]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75190) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 928.292949] env[75190]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 928.293049] env[75190]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 928.293914] env[75190]: WARNING oslo_policy.policy [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 928.294128] env[75190]: WARNING oslo_policy.policy [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 928.295681] env[75190]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 928.312562] env[75190]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Session token - acquiring [ 928.312778] env[75190]: DEBUG networking_nsxv3.common.locking [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75190) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 928.312983] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 928.313118] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 928.313779] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 928.322726] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 928.322982] env[75190]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 928.323131] env[75190]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75190) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 930.329173] env[75190]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Session token - acquiring [ 930.329500] env[75190]: DEBUG networking_nsxv3.common.locking [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75190) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 930.329697] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 930.329825] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 930.330097] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 930.339854] env[75190]: DEBUG oslo_concurrency.lockutils [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75190) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 930.340093] env[75190]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 930.340240] env[75190]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75190) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 932.343130] env[75190]: CRITICAL neutron [None req-b6cc9f95-39a0-495e-ae70-1e8589b6e570 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 932.343130] env[75190]: ERROR neutron Traceback (most recent call last): [ 932.343130] env[75190]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 932.343130] env[75190]: ERROR neutron sys.exit(main()) [ 932.343130] env[75190]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 932.343130] env[75190]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 932.343130] env[75190]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 932.343130] env[75190]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 932.343130] env[75190]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 932.343130] env[75190]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 932.343130] env[75190]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 932.343130] env[75190]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 932.343130] env[75190]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 932.343130] env[75190]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 932.343130] env[75190]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 932.343130] env[75190]: ERROR neutron response = self.get(path=path, params=params) [ 932.343980] env[75190]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 932.343980] env[75190]: ERROR neutron raise RuntimeError(msg, last_err) [ 932.343980] env[75190]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 932.343980] env[75190]: ERROR neutron [ 934.969897] env[75322]: INFO neutron.common.config [-] Logging enabled! [ 934.969897] env[75322]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 934.969897] env[75322]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75322) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 934.970321] env[75322]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 934.970464] env[75322]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 934.971477] env[75322]: WARNING oslo_policy.policy [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 934.971636] env[75322]: WARNING oslo_policy.policy [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 934.972945] env[75322]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 934.993280] env[75322]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Session token - acquiring [ 934.993280] env[75322]: DEBUG networking_nsxv3.common.locking [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75322) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 934.993280] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 934.993844] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 934.993844] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 935.001537] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 935.001762] env[75322]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 935.001899] env[75322]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75322) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 937.008136] env[75322]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Session token - acquiring [ 937.008478] env[75322]: DEBUG networking_nsxv3.common.locking [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75322) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 937.008664] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 937.008782] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 937.009053] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 937.018373] env[75322]: DEBUG oslo_concurrency.lockutils [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75322) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 937.018678] env[75322]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 937.018875] env[75322]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75322) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 939.020929] env[75322]: CRITICAL neutron [None req-0eedf3bf-396f-4dc9-936d-86b1e3f57855 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 939.020929] env[75322]: ERROR neutron Traceback (most recent call last): [ 939.020929] env[75322]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 939.020929] env[75322]: ERROR neutron sys.exit(main()) [ 939.020929] env[75322]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 939.020929] env[75322]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 939.020929] env[75322]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 939.020929] env[75322]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 939.020929] env[75322]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 939.020929] env[75322]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 939.020929] env[75322]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 939.020929] env[75322]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 939.020929] env[75322]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 939.020929] env[75322]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 939.020929] env[75322]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 939.020929] env[75322]: ERROR neutron response = self.get(path=path, params=params) [ 939.021994] env[75322]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 939.021994] env[75322]: ERROR neutron raise RuntimeError(msg, last_err) [ 939.021994] env[75322]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 939.021994] env[75322]: ERROR neutron [ 941.724968] env[75437]: INFO neutron.common.config [-] Logging enabled! [ 941.725279] env[75437]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 941.725532] env[75437]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75437) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 941.725765] env[75437]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 941.725858] env[75437]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 941.726747] env[75437]: WARNING oslo_policy.policy [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 941.726973] env[75437]: WARNING oslo_policy.policy [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 941.728273] env[75437]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 941.741998] env[75437]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Session token - acquiring [ 941.742796] env[75437]: DEBUG networking_nsxv3.common.locking [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75437) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 941.743001] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 941.743127] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 941.743493] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 941.752906] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 941.753166] env[75437]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 941.753317] env[75437]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75437) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 943.761403] env[75437]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Session token - acquiring [ 943.761756] env[75437]: DEBUG networking_nsxv3.common.locking [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75437) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 943.761980] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 943.768532] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 943.768532] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 943.771502] env[75437]: DEBUG oslo_concurrency.lockutils [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75437) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 943.771755] env[75437]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 943.771911] env[75437]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75437) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 945.773013] env[75437]: CRITICAL neutron [None req-c92070ce-056b-4c4a-9857-960824fe11ab None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 945.773013] env[75437]: ERROR neutron Traceback (most recent call last): [ 945.773013] env[75437]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 945.773013] env[75437]: ERROR neutron sys.exit(main()) [ 945.773013] env[75437]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 945.773013] env[75437]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 945.773013] env[75437]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 945.773013] env[75437]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 945.773013] env[75437]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 945.773013] env[75437]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 945.773013] env[75437]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 945.773013] env[75437]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 945.773013] env[75437]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 945.773013] env[75437]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 945.773013] env[75437]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 945.773013] env[75437]: ERROR neutron response = self.get(path=path, params=params) [ 945.773786] env[75437]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 945.773786] env[75437]: ERROR neutron raise RuntimeError(msg, last_err) [ 945.773786] env[75437]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 945.773786] env[75437]: ERROR neutron [ 948.340127] env[75550]: INFO neutron.common.config [-] Logging enabled! [ 948.340535] env[75550]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 948.340661] env[75550]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75550) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 948.340885] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 948.340978] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 948.341823] env[75550]: WARNING oslo_policy.policy [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 948.342019] env[75550]: WARNING oslo_policy.policy [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 948.343168] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 948.357193] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Session token - acquiring [ 948.357960] env[75550]: DEBUG networking_nsxv3.common.locking [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75550) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 948.358155] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 948.358324] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 948.358697] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 948.367542] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 948.367783] env[75550]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 948.367929] env[75550]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75550) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 950.374153] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Session token - acquiring [ 950.374593] env[75550]: DEBUG networking_nsxv3.common.locking [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75550) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 950.374792] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 950.374905] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 950.375177] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 950.385378] env[75550]: DEBUG oslo_concurrency.lockutils [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 950.385378] env[75550]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 950.385378] env[75550]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75550) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 952.388124] env[75550]: CRITICAL neutron [None req-c8e9780f-b587-4fcf-a65e-621df1bf080b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 952.388124] env[75550]: ERROR neutron Traceback (most recent call last): [ 952.388124] env[75550]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 952.388124] env[75550]: ERROR neutron sys.exit(main()) [ 952.388124] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 952.388124] env[75550]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 952.388124] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 952.388124] env[75550]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 952.388124] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 952.388124] env[75550]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 952.388124] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 952.388124] env[75550]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 952.388124] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 952.388124] env[75550]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 952.388124] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 952.388124] env[75550]: ERROR neutron response = self.get(path=path, params=params) [ 952.388885] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 952.388885] env[75550]: ERROR neutron raise RuntimeError(msg, last_err) [ 952.388885] env[75550]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 952.388885] env[75550]: ERROR neutron [ 954.991142] env[75580]: INFO neutron.common.config [-] Logging enabled! [ 954.991486] env[75580]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 954.991675] env[75580]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75580) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 954.991910] env[75580]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 954.991994] env[75580]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 954.992805] env[75580]: WARNING oslo_policy.policy [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 954.993022] env[75580]: WARNING oslo_policy.policy [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 954.994213] env[75580]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 955.009217] env[75580]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Session token - acquiring [ 955.009973] env[75580]: DEBUG networking_nsxv3.common.locking [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75580) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 955.010188] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 955.010315] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 955.010672] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 955.019549] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 955.019799] env[75580]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 955.019943] env[75580]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75580) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 957.026038] env[75580]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Session token - acquiring [ 957.026372] env[75580]: DEBUG networking_nsxv3.common.locking [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75580) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 957.026592] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 957.026678] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 957.026945] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 957.036024] env[75580]: DEBUG oslo_concurrency.lockutils [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75580) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 957.036305] env[75580]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 957.036446] env[75580]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75580) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 959.039378] env[75580]: CRITICAL neutron [None req-8e8272cf-1721-435a-86c7-933ce2ddc1ea None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 959.039378] env[75580]: ERROR neutron Traceback (most recent call last): [ 959.039378] env[75580]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 959.039378] env[75580]: ERROR neutron sys.exit(main()) [ 959.039378] env[75580]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 959.039378] env[75580]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 959.039378] env[75580]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 959.039378] env[75580]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 959.039378] env[75580]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 959.039378] env[75580]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 959.039378] env[75580]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 959.039378] env[75580]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 959.039378] env[75580]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 959.039378] env[75580]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 959.039378] env[75580]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 959.039378] env[75580]: ERROR neutron response = self.get(path=path, params=params) [ 959.040123] env[75580]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 959.040123] env[75580]: ERROR neutron raise RuntimeError(msg, last_err) [ 959.040123] env[75580]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 959.040123] env[75580]: ERROR neutron [ 961.430807] env[75627]: INFO neutron.common.config [-] Logging enabled! [ 961.431145] env[75627]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 961.431330] env[75627]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75627) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 961.431567] env[75627]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 961.431657] env[75627]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 961.432513] env[75627]: WARNING oslo_policy.policy [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 961.432733] env[75627]: WARNING oslo_policy.policy [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 961.433953] env[75627]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 961.447823] env[75627]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Session token - acquiring [ 961.448637] env[75627]: DEBUG networking_nsxv3.common.locking [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75627) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 961.448849] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 961.448976] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 961.449342] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 961.458560] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 961.458560] env[75627]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 961.458689] env[75627]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75627) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 963.466008] env[75627]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Session token - acquiring [ 963.466304] env[75627]: DEBUG networking_nsxv3.common.locking [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75627) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 963.466499] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 963.466626] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 963.466909] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 963.477458] env[75627]: DEBUG oslo_concurrency.lockutils [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75627) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 963.477714] env[75627]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 963.477855] env[75627]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75627) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 965.481321] env[75627]: CRITICAL neutron [None req-f0d4ccdf-ee70-42b4-9a5a-a358e0207043 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 965.481321] env[75627]: ERROR neutron Traceback (most recent call last): [ 965.481321] env[75627]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 965.481321] env[75627]: ERROR neutron sys.exit(main()) [ 965.481321] env[75627]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 965.481321] env[75627]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 965.481321] env[75627]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 965.481321] env[75627]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 965.481321] env[75627]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 965.481321] env[75627]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 965.481321] env[75627]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 965.481321] env[75627]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 965.481321] env[75627]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 965.481321] env[75627]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 965.481321] env[75627]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 965.481321] env[75627]: ERROR neutron response = self.get(path=path, params=params) [ 965.482489] env[75627]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 965.482489] env[75627]: ERROR neutron raise RuntimeError(msg, last_err) [ 965.482489] env[75627]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 965.482489] env[75627]: ERROR neutron [ 968.031919] env[75731]: INFO neutron.common.config [-] Logging enabled! [ 968.032320] env[75731]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 968.032481] env[75731]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75731) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 968.032707] env[75731]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 968.032801] env[75731]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 968.033847] env[75731]: WARNING oslo_policy.policy [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 968.033847] env[75731]: WARNING oslo_policy.policy [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 968.035021] env[75731]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 968.048490] env[75731]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Session token - acquiring [ 968.049256] env[75731]: DEBUG networking_nsxv3.common.locking [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75731) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 968.049454] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 968.049582] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 968.049939] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 968.059317] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 968.059561] env[75731]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 968.059704] env[75731]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75731) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 970.065820] env[75731]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Session token - acquiring [ 970.066434] env[75731]: DEBUG networking_nsxv3.common.locking [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75731) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 970.066539] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 970.066674] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 970.066968] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 970.076816] env[75731]: DEBUG oslo_concurrency.lockutils [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75731) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 970.077095] env[75731]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 970.077246] env[75731]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75731) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 972.080179] env[75731]: CRITICAL neutron [None req-b0770ac7-dbbf-4943-aff7-7cdf177fe558 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 972.080179] env[75731]: ERROR neutron Traceback (most recent call last): [ 972.080179] env[75731]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 972.080179] env[75731]: ERROR neutron sys.exit(main()) [ 972.080179] env[75731]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 972.080179] env[75731]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 972.080179] env[75731]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 972.080179] env[75731]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 972.080179] env[75731]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 972.080179] env[75731]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 972.080179] env[75731]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 972.080179] env[75731]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 972.080179] env[75731]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 972.080179] env[75731]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 972.080179] env[75731]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 972.080179] env[75731]: ERROR neutron response = self.get(path=path, params=params) [ 972.080922] env[75731]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 972.080922] env[75731]: ERROR neutron raise RuntimeError(msg, last_err) [ 972.080922] env[75731]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 972.080922] env[75731]: ERROR neutron [ 974.716175] env[75790]: INFO neutron.common.config [-] Logging enabled! [ 974.716543] env[75790]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 974.716724] env[75790]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75790) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 974.716960] env[75790]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 974.717078] env[75790]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 974.717914] env[75790]: WARNING oslo_policy.policy [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 974.718136] env[75790]: WARNING oslo_policy.policy [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 974.719358] env[75790]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 974.734183] env[75790]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Session token - acquiring [ 974.734982] env[75790]: DEBUG networking_nsxv3.common.locking [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75790) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 974.735196] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 974.735326] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 974.735715] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 974.744741] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 974.744993] env[75790]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 974.745143] env[75790]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75790) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 976.751330] env[75790]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Session token - acquiring [ 976.751701] env[75790]: DEBUG networking_nsxv3.common.locking [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75790) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 976.751701] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 976.751840] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 976.752101] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 976.761671] env[75790]: DEBUG oslo_concurrency.lockutils [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75790) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 976.761928] env[75790]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 976.762069] env[75790]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75790) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 978.764954] env[75790]: CRITICAL neutron [None req-f6c6f2e1-cf17-4ab1-959e-37b8afad771a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 978.764954] env[75790]: ERROR neutron Traceback (most recent call last): [ 978.764954] env[75790]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 978.764954] env[75790]: ERROR neutron sys.exit(main()) [ 978.764954] env[75790]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 978.764954] env[75790]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 978.764954] env[75790]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 978.764954] env[75790]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 978.764954] env[75790]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 978.764954] env[75790]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 978.764954] env[75790]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 978.764954] env[75790]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 978.764954] env[75790]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 978.764954] env[75790]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 978.764954] env[75790]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 978.764954] env[75790]: ERROR neutron response = self.get(path=path, params=params) [ 978.765661] env[75790]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 978.765661] env[75790]: ERROR neutron raise RuntimeError(msg, last_err) [ 978.765661] env[75790]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 978.765661] env[75790]: ERROR neutron [ 981.467684] env[75852]: INFO neutron.common.config [-] Logging enabled! [ 981.468108] env[75852]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 981.468416] env[75852]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75852) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 981.468723] env[75852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 981.468867] env[75852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 981.469961] env[75852]: WARNING oslo_policy.policy [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 981.470346] env[75852]: WARNING oslo_policy.policy [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 981.471919] env[75852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 981.497079] env[75852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Session token - acquiring [ 981.498000] env[75852]: DEBUG networking_nsxv3.common.locking [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75852) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 981.498224] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 981.498381] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 981.498786] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 981.513810] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 981.514314] env[75852]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 981.514314] env[75852]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75852) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 983.520123] env[75852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Session token - acquiring [ 983.520475] env[75852]: DEBUG networking_nsxv3.common.locking [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75852) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 983.520724] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 983.520865] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 983.521144] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 983.530540] env[75852]: DEBUG oslo_concurrency.lockutils [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 983.530770] env[75852]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 983.530906] env[75852]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75852) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 985.532899] env[75852]: CRITICAL neutron [None req-04e9eb78-d645-4fa6-882c-87c2a62ac8d5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 985.532899] env[75852]: ERROR neutron Traceback (most recent call last): [ 985.532899] env[75852]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 985.532899] env[75852]: ERROR neutron sys.exit(main()) [ 985.532899] env[75852]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 985.532899] env[75852]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 985.532899] env[75852]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 985.532899] env[75852]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 985.532899] env[75852]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 985.532899] env[75852]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 985.532899] env[75852]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 985.532899] env[75852]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 985.532899] env[75852]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 985.532899] env[75852]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 985.532899] env[75852]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 985.532899] env[75852]: ERROR neutron response = self.get(path=path, params=params) [ 985.533554] env[75852]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 985.533554] env[75852]: ERROR neutron raise RuntimeError(msg, last_err) [ 985.533554] env[75852]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 985.533554] env[75852]: ERROR neutron [ 988.202025] env[75927]: INFO neutron.common.config [-] Logging enabled! [ 988.202357] env[75927]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 988.202531] env[75927]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75927) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 988.202762] env[75927]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 988.202854] env[75927]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 988.203680] env[75927]: WARNING oslo_policy.policy [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 988.203901] env[75927]: WARNING oslo_policy.policy [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 988.205204] env[75927]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 988.219429] env[75927]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Session token - acquiring [ 988.220235] env[75927]: DEBUG networking_nsxv3.common.locking [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75927) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 988.220438] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 988.220564] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 988.220926] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 988.229707] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 988.229952] env[75927]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 988.230107] env[75927]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75927) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 990.236989] env[75927]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Session token - acquiring [ 990.237424] env[75927]: DEBUG networking_nsxv3.common.locking [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75927) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 990.237660] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 990.237869] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 990.238253] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 990.251600] env[75927]: DEBUG oslo_concurrency.lockutils [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75927) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 990.251872] env[75927]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 990.252113] env[75927]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75927) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 992.255156] env[75927]: CRITICAL neutron [None req-c0adbc22-f227-493f-89ca-0bd6b9772da2 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 992.255156] env[75927]: ERROR neutron Traceback (most recent call last): [ 992.255156] env[75927]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 992.255156] env[75927]: ERROR neutron sys.exit(main()) [ 992.255156] env[75927]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 992.255156] env[75927]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 992.255156] env[75927]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 992.255156] env[75927]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 992.255156] env[75927]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 992.255156] env[75927]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 992.255156] env[75927]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 992.255156] env[75927]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 992.255156] env[75927]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 992.255156] env[75927]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 992.255156] env[75927]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 992.255156] env[75927]: ERROR neutron response = self.get(path=path, params=params) [ 992.256170] env[75927]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 992.256170] env[75927]: ERROR neutron raise RuntimeError(msg, last_err) [ 992.256170] env[75927]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 992.256170] env[75927]: ERROR neutron [ 994.755912] env[75967]: INFO neutron.common.config [-] Logging enabled! [ 994.756250] env[75967]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 994.756547] env[75967]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75967) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 994.756778] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 994.756873] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 994.757751] env[75967]: WARNING oslo_policy.policy [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 994.757973] env[75967]: WARNING oslo_policy.policy [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 994.759224] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 994.779451] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Session token - acquiring [ 994.780414] env[75967]: DEBUG networking_nsxv3.common.locking [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75967) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 994.780632] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 994.780761] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 994.781166] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 994.791408] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 994.791696] env[75967]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 994.791846] env[75967]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75967) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 996.797971] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Session token - acquiring [ 996.798310] env[75967]: DEBUG networking_nsxv3.common.locking [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75967) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 996.798523] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 996.798652] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 996.798934] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 996.808663] env[75967]: DEBUG oslo_concurrency.lockutils [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 996.808955] env[75967]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 996.809145] env[75967]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75967) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 998.812049] env[75967]: CRITICAL neutron [None req-ff54e083-3bbc-4a6a-a8b1-d6df4b41c642 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 998.812049] env[75967]: ERROR neutron Traceback (most recent call last): [ 998.812049] env[75967]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 998.812049] env[75967]: ERROR neutron sys.exit(main()) [ 998.812049] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 998.812049] env[75967]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 998.812049] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 998.812049] env[75967]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 998.812049] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 998.812049] env[75967]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 998.812049] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 998.812049] env[75967]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 998.812049] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 998.812049] env[75967]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 998.812049] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 998.812049] env[75967]: ERROR neutron response = self.get(path=path, params=params) [ 998.812999] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 998.812999] env[75967]: ERROR neutron raise RuntimeError(msg, last_err) [ 998.812999] env[75967]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 998.812999] env[75967]: ERROR neutron [ 1001.355262] env[76116]: INFO neutron.common.config [-] Logging enabled! [ 1001.355664] env[76116]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 1001.355775] env[76116]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76116) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1001.356071] env[76116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1001.356187] env[76116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1001.357048] env[76116]: WARNING oslo_policy.policy [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1001.357276] env[76116]: WARNING oslo_policy.policy [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1001.358447] env[76116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1001.372636] env[76116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Session token - acquiring [ 1001.373368] env[76116]: DEBUG networking_nsxv3.common.locking [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76116) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1001.373567] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1001.373692] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1001.374085] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1001.382900] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1001.383216] env[76116]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1001.383370] env[76116]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76116) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1003.389757] env[76116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Session token - acquiring [ 1003.390344] env[76116]: DEBUG networking_nsxv3.common.locking [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76116) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1003.390564] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1003.390694] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1003.390987] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1003.399849] env[76116]: DEBUG oslo_concurrency.lockutils [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1003.400102] env[76116]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1003.400249] env[76116]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76116) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1005.403199] env[76116]: CRITICAL neutron [None req-5786d661-90ab-492a-9ba6-2cce0e659ca0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1005.403199] env[76116]: ERROR neutron Traceback (most recent call last): [ 1005.403199] env[76116]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1005.403199] env[76116]: ERROR neutron sys.exit(main()) [ 1005.403199] env[76116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1005.403199] env[76116]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1005.403199] env[76116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1005.403199] env[76116]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1005.403199] env[76116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1005.403199] env[76116]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1005.403199] env[76116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1005.403199] env[76116]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1005.403199] env[76116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1005.403199] env[76116]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1005.403199] env[76116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1005.403199] env[76116]: ERROR neutron response = self.get(path=path, params=params) [ 1005.404071] env[76116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1005.404071] env[76116]: ERROR neutron raise RuntimeError(msg, last_err) [ 1005.404071] env[76116]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1005.404071] env[76116]: ERROR neutron [ 1007.937376] env[76153]: INFO neutron.common.config [-] Logging enabled! [ 1007.937813] env[76153]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 1007.937876] env[76153]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76153) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1007.938106] env[76153]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1007.938203] env[76153]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1007.939034] env[76153]: WARNING oslo_policy.policy [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1007.939271] env[76153]: WARNING oslo_policy.policy [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1007.940464] env[76153]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1007.955207] env[76153]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Session token - acquiring [ 1007.955976] env[76153]: DEBUG networking_nsxv3.common.locking [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76153) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1007.956213] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1007.956344] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1007.956748] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1007.965739] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1007.965997] env[76153]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1007.966159] env[76153]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76153) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1009.971995] env[76153]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Session token - acquiring [ 1009.972377] env[76153]: DEBUG networking_nsxv3.common.locking [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76153) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1009.972598] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1009.972725] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1009.973059] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1009.982569] env[76153]: DEBUG oslo_concurrency.lockutils [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76153) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1009.982829] env[76153]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1009.982993] env[76153]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76153) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1011.984883] env[76153]: CRITICAL neutron [None req-f1f58704-fd70-4271-be70-4e89c7b64d29 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1011.984883] env[76153]: ERROR neutron Traceback (most recent call last): [ 1011.984883] env[76153]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1011.984883] env[76153]: ERROR neutron sys.exit(main()) [ 1011.984883] env[76153]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1011.984883] env[76153]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1011.984883] env[76153]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1011.984883] env[76153]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1011.984883] env[76153]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1011.984883] env[76153]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1011.984883] env[76153]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1011.984883] env[76153]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1011.984883] env[76153]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1011.984883] env[76153]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1011.984883] env[76153]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1011.984883] env[76153]: ERROR neutron response = self.get(path=path, params=params) [ 1011.985579] env[76153]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1011.985579] env[76153]: ERROR neutron raise RuntimeError(msg, last_err) [ 1011.985579] env[76153]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1011.985579] env[76153]: ERROR neutron [ 1014.426590] env[76204]: INFO neutron.common.config [-] Logging enabled! [ 1014.427011] env[76204]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 1014.427119] env[76204]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76204) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1014.427350] env[76204]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1014.427443] env[76204]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1014.428327] env[76204]: WARNING oslo_policy.policy [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1014.428547] env[76204]: WARNING oslo_policy.policy [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1014.429761] env[76204]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1014.444210] env[76204]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Session token - acquiring [ 1014.444982] env[76204]: DEBUG networking_nsxv3.common.locking [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76204) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1014.445186] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1014.445318] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1014.445676] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1014.454564] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1014.454812] env[76204]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1014.454956] env[76204]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76204) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1016.461264] env[76204]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Session token - acquiring [ 1016.461624] env[76204]: DEBUG networking_nsxv3.common.locking [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76204) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1016.461837] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1016.461986] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1016.462244] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1016.471306] env[76204]: DEBUG oslo_concurrency.lockutils [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76204) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1016.471542] env[76204]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1016.471683] env[76204]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76204) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1018.474670] env[76204]: CRITICAL neutron [None req-e97a8276-d0d0-46c3-a3ef-4157a83fe167 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1018.474670] env[76204]: ERROR neutron Traceback (most recent call last): [ 1018.474670] env[76204]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1018.474670] env[76204]: ERROR neutron sys.exit(main()) [ 1018.474670] env[76204]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1018.474670] env[76204]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1018.474670] env[76204]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1018.474670] env[76204]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1018.474670] env[76204]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1018.474670] env[76204]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1018.474670] env[76204]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1018.474670] env[76204]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1018.474670] env[76204]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1018.474670] env[76204]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1018.474670] env[76204]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1018.474670] env[76204]: ERROR neutron response = self.get(path=path, params=params) [ 1018.475482] env[76204]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1018.475482] env[76204]: ERROR neutron raise RuntimeError(msg, last_err) [ 1018.475482] env[76204]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1018.475482] env[76204]: ERROR neutron [ 1020.873853] env[76225]: INFO neutron.common.config [-] Logging enabled! [ 1020.874231] env[76225]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 1020.874351] env[76225]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76225) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1020.874568] env[76225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1020.874658] env[76225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1020.875518] env[76225]: WARNING oslo_policy.policy [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1020.875744] env[76225]: WARNING oslo_policy.policy [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1020.877044] env[76225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1020.891134] env[76225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Session token - acquiring [ 1020.891905] env[76225]: DEBUG networking_nsxv3.common.locking [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76225) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1020.892122] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1020.892254] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1020.892644] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1020.901456] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1020.901704] env[76225]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1020.901851] env[76225]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76225) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1022.907858] env[76225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Session token - acquiring [ 1022.908327] env[76225]: DEBUG networking_nsxv3.common.locking [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76225) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1022.908494] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1022.908618] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1022.908890] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1022.917965] env[76225]: DEBUG oslo_concurrency.lockutils [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1022.918193] env[76225]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1022.918333] env[76225]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76225) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1024.921242] env[76225]: CRITICAL neutron [None req-238bd89e-be82-4a9f-8b18-ec9b32be5c56 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1024.921242] env[76225]: ERROR neutron Traceback (most recent call last): [ 1024.921242] env[76225]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1024.921242] env[76225]: ERROR neutron sys.exit(main()) [ 1024.921242] env[76225]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1024.921242] env[76225]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1024.921242] env[76225]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1024.921242] env[76225]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1024.921242] env[76225]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1024.921242] env[76225]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1024.921242] env[76225]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1024.921242] env[76225]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1024.921242] env[76225]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1024.921242] env[76225]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1024.921242] env[76225]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1024.921242] env[76225]: ERROR neutron response = self.get(path=path, params=params) [ 1024.922032] env[76225]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1024.922032] env[76225]: ERROR neutron raise RuntimeError(msg, last_err) [ 1024.922032] env[76225]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1024.922032] env[76225]: ERROR neutron [ 1027.371597] env[76232]: INFO neutron.common.config [-] Logging enabled! [ 1027.372113] env[76232]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 1027.372177] env[76232]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76232) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1027.372366] env[76232]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1027.372455] env[76232]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1027.373275] env[76232]: WARNING oslo_policy.policy [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1027.373493] env[76232]: WARNING oslo_policy.policy [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1027.374683] env[76232]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1027.388998] env[76232]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Session token - acquiring [ 1027.389780] env[76232]: DEBUG networking_nsxv3.common.locking [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76232) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1027.389986] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1027.390111] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1027.390732] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1027.399651] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1027.399889] env[76232]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1027.400057] env[76232]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76232) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1029.407263] env[76232]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Session token - acquiring [ 1029.407797] env[76232]: DEBUG networking_nsxv3.common.locking [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76232) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1029.408179] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1029.408453] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1029.408879] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1029.418750] env[76232]: DEBUG oslo_concurrency.lockutils [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76232) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1029.419001] env[76232]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1029.419136] env[76232]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76232) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1031.422066] env[76232]: CRITICAL neutron [None req-c71c757b-ce70-44d9-b3a9-510af7100f53 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1031.422066] env[76232]: ERROR neutron Traceback (most recent call last): [ 1031.422066] env[76232]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1031.422066] env[76232]: ERROR neutron sys.exit(main()) [ 1031.422066] env[76232]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1031.422066] env[76232]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1031.422066] env[76232]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1031.422066] env[76232]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1031.422066] env[76232]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1031.422066] env[76232]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1031.422066] env[76232]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1031.422066] env[76232]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1031.422066] env[76232]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1031.422066] env[76232]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1031.422066] env[76232]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1031.422066] env[76232]: ERROR neutron response = self.get(path=path, params=params) [ 1031.423051] env[76232]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1031.423051] env[76232]: ERROR neutron raise RuntimeError(msg, last_err) [ 1031.423051] env[76232]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1031.423051] env[76232]: ERROR neutron [ 1033.754015] env[76311]: INFO neutron.common.config [-] Logging enabled! [ 1033.754380] env[76311]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev63 [ 1033.754483] env[76311]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76311) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1033.754689] env[76311]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1033.754778] env[76311]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1033.755594] env[76311]: WARNING oslo_policy.policy [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1033.755831] env[76311]: WARNING oslo_policy.policy [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1033.757058] env[76311]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1033.771147] env[76311]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Session token - acquiring [ 1033.771901] env[76311]: DEBUG networking_nsxv3.common.locking [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76311) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1033.772149] env[76311]: DEBUG oslo_concurrency.lockutils [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76311) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1033.772277] env[76311]: DEBUG oslo_concurrency.lockutils [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76311) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1033.772631] env[76311]: DEBUG oslo_concurrency.lockutils [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76311) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1033.782507] env[76311]: DEBUG oslo_concurrency.lockutils [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76311) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1033.782745] env[76311]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1033.782887] env[76311]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-66202bec-6951-46ca-bae6-a279917daf78 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76311) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}