[ 463.759367] env[61269]: INFO neutron.common.config [-] Logging enabled! [ 463.759855] env[61269]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 463.759905] env[61269]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61269) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 463.760119] env[61269]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 463.760210] env[61269]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 463.761060] env[61269]: WARNING oslo_policy.policy [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 463.761276] env[61269]: WARNING oslo_policy.policy [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 463.762461] env[61269]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 463.776427] env[61269]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Session token - acquiring [ 463.777190] env[61269]: DEBUG networking_nsxv3.common.locking [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61269) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 463.777389] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 463.777518] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 463.777924] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 463.787325] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 463.787567] env[61269]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 463.787714] env[61269]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61269) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 465.793922] env[61269]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Session token - acquiring [ 465.794444] env[61269]: DEBUG networking_nsxv3.common.locking [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61269) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 465.794444] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 465.794540] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 465.794730] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 465.803600] env[61269]: DEBUG oslo_concurrency.lockutils [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61269) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 465.803871] env[61269]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 465.804058] env[61269]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61269) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 467.806986] env[61269]: CRITICAL neutron [None req-5697b575-c6d7-40e1-ba87-76178a631c2b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 467.806986] env[61269]: ERROR neutron Traceback (most recent call last): [ 467.806986] env[61269]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 467.806986] env[61269]: ERROR neutron sys.exit(main()) [ 467.806986] env[61269]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 467.806986] env[61269]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 467.806986] env[61269]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 467.806986] env[61269]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 467.806986] env[61269]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 467.806986] env[61269]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 467.806986] env[61269]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 467.806986] env[61269]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 467.806986] env[61269]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 467.806986] env[61269]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 467.806986] env[61269]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 467.806986] env[61269]: ERROR neutron response = self.get(path=path, params=params) [ 467.807802] env[61269]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 467.807802] env[61269]: ERROR neutron raise RuntimeError(msg, last_err) [ 467.807802] env[61269]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 467.807802] env[61269]: ERROR neutron [ 470.359301] env[61302]: INFO neutron.common.config [-] Logging enabled! [ 470.359732] env[61302]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 470.359803] env[61302]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61302) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 470.360035] env[61302]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 470.360136] env[61302]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 470.360940] env[61302]: WARNING oslo_policy.policy [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 470.361169] env[61302]: WARNING oslo_policy.policy [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 470.362323] env[61302]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 470.376718] env[61302]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Session token - acquiring [ 470.377505] env[61302]: DEBUG networking_nsxv3.common.locking [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61302) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 470.377686] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 470.377823] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 470.378187] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 470.387243] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 470.387485] env[61302]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 470.387645] env[61302]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61302) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 472.393780] env[61302]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Session token - acquiring [ 472.394214] env[61302]: DEBUG networking_nsxv3.common.locking [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61302) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 472.394308] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 472.394391] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 472.394672] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 472.403886] env[61302]: DEBUG oslo_concurrency.lockutils [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61302) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 472.404123] env[61302]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 472.404267] env[61302]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61302) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 474.407205] env[61302]: CRITICAL neutron [None req-f56f93bc-555e-48fe-8296-bd61ffa58ccb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 474.407205] env[61302]: ERROR neutron Traceback (most recent call last): [ 474.407205] env[61302]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 474.407205] env[61302]: ERROR neutron sys.exit(main()) [ 474.407205] env[61302]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 474.407205] env[61302]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 474.407205] env[61302]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 474.407205] env[61302]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 474.407205] env[61302]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 474.407205] env[61302]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 474.407205] env[61302]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 474.407205] env[61302]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 474.407205] env[61302]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 474.407205] env[61302]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 474.407205] env[61302]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 474.407205] env[61302]: ERROR neutron response = self.get(path=path, params=params) [ 474.408253] env[61302]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 474.408253] env[61302]: ERROR neutron raise RuntimeError(msg, last_err) [ 474.408253] env[61302]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 474.408253] env[61302]: ERROR neutron [ 476.870544] env[61325]: INFO neutron.common.config [-] Logging enabled! [ 476.870926] env[61325]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 476.871068] env[61325]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61325) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 476.871305] env[61325]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 476.871403] env[61325]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 476.872368] env[61325]: WARNING oslo_policy.policy [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 476.872620] env[61325]: WARNING oslo_policy.policy [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 476.873857] env[61325]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 476.888330] env[61325]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Session token - acquiring [ 476.889155] env[61325]: DEBUG networking_nsxv3.common.locking [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61325) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 476.889362] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 476.889566] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 476.889949] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 476.899002] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 476.899265] env[61325]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 476.899413] env[61325]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61325) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 478.905792] env[61325]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Session token - acquiring [ 478.906111] env[61325]: DEBUG networking_nsxv3.common.locking [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61325) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 478.906276] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 478.906400] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 478.906674] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 478.915554] env[61325]: DEBUG oslo_concurrency.lockutils [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61325) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 478.915787] env[61325]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 478.915927] env[61325]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61325) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 480.917734] env[61325]: CRITICAL neutron [None req-31e9dc02-70cf-49bf-88ee-2e2eac7aebc1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 480.917734] env[61325]: ERROR neutron Traceback (most recent call last): [ 480.917734] env[61325]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 480.917734] env[61325]: ERROR neutron sys.exit(main()) [ 480.917734] env[61325]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 480.917734] env[61325]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 480.917734] env[61325]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 480.917734] env[61325]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 480.917734] env[61325]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 480.917734] env[61325]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 480.917734] env[61325]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 480.917734] env[61325]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 480.917734] env[61325]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 480.917734] env[61325]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 480.917734] env[61325]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 480.917734] env[61325]: ERROR neutron response = self.get(path=path, params=params) [ 480.918498] env[61325]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 480.918498] env[61325]: ERROR neutron raise RuntimeError(msg, last_err) [ 480.918498] env[61325]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 480.918498] env[61325]: ERROR neutron [ 483.344369] env[61372]: INFO neutron.common.config [-] Logging enabled! [ 483.344756] env[61372]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 483.344851] env[61372]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61372) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 483.345074] env[61372]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 483.345166] env[61372]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 483.345965] env[61372]: WARNING oslo_policy.policy [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 483.346177] env[61372]: WARNING oslo_policy.policy [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 483.347315] env[61372]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 483.361570] env[61372]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Session token - acquiring [ 483.362290] env[61372]: DEBUG networking_nsxv3.common.locking [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61372) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 483.362510] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 483.362612] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 483.362958] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 483.371727] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 483.371962] env[61372]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 483.372118] env[61372]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61372) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 485.378150] env[61372]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Session token - acquiring [ 485.378546] env[61372]: DEBUG networking_nsxv3.common.locking [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61372) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 485.378985] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 485.379081] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 485.379290] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 485.388412] env[61372]: DEBUG oslo_concurrency.lockutils [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61372) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 485.388637] env[61372]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 485.388777] env[61372]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61372) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 487.391758] env[61372]: CRITICAL neutron [None req-d5a3cb16-6cde-4529-91c3-f8d22b0f42b4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 487.391758] env[61372]: ERROR neutron Traceback (most recent call last): [ 487.391758] env[61372]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 487.391758] env[61372]: ERROR neutron sys.exit(main()) [ 487.391758] env[61372]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 487.391758] env[61372]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 487.391758] env[61372]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 487.391758] env[61372]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 487.391758] env[61372]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 487.391758] env[61372]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 487.391758] env[61372]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 487.391758] env[61372]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 487.391758] env[61372]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 487.391758] env[61372]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 487.391758] env[61372]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 487.391758] env[61372]: ERROR neutron response = self.get(path=path, params=params) [ 487.392622] env[61372]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 487.392622] env[61372]: ERROR neutron raise RuntimeError(msg, last_err) [ 487.392622] env[61372]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 487.392622] env[61372]: ERROR neutron [ 489.817863] env[61377]: INFO neutron.common.config [-] Logging enabled! [ 489.818285] env[61377]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 489.818379] env[61377]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61377) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 489.818562] env[61377]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 489.818655] env[61377]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 489.819451] env[61377]: WARNING oslo_policy.policy [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 489.819696] env[61377]: WARNING oslo_policy.policy [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 489.820895] env[61377]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 489.834446] env[61377]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Session token - acquiring [ 489.835171] env[61377]: DEBUG networking_nsxv3.common.locking [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61377) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 489.835369] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 489.835496] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 489.836053] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 489.844516] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 489.844765] env[61377]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 489.844907] env[61377]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61377) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 491.852889] env[61377]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Session token - acquiring [ 491.853240] env[61377]: DEBUG networking_nsxv3.common.locking [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61377) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 491.853405] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 491.853504] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 491.853819] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 491.862590] env[61377]: DEBUG oslo_concurrency.lockutils [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61377) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 491.862825] env[61377]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 491.862968] env[61377]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61377) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 493.865921] env[61377]: CRITICAL neutron [None req-40d0065d-01b1-4259-b29e-166b3542307b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 493.865921] env[61377]: ERROR neutron Traceback (most recent call last): [ 493.865921] env[61377]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 493.865921] env[61377]: ERROR neutron sys.exit(main()) [ 493.865921] env[61377]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 493.865921] env[61377]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 493.865921] env[61377]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 493.865921] env[61377]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 493.865921] env[61377]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 493.865921] env[61377]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 493.865921] env[61377]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 493.865921] env[61377]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 493.865921] env[61377]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 493.865921] env[61377]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 493.865921] env[61377]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 493.865921] env[61377]: ERROR neutron response = self.get(path=path, params=params) [ 493.866754] env[61377]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 493.866754] env[61377]: ERROR neutron raise RuntimeError(msg, last_err) [ 493.866754] env[61377]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 493.866754] env[61377]: ERROR neutron [ 496.318963] env[61427]: INFO neutron.common.config [-] Logging enabled! [ 496.319429] env[61427]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 496.319505] env[61427]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61427) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 496.319740] env[61427]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 496.319826] env[61427]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 496.320626] env[61427]: WARNING oslo_policy.policy [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 496.320848] env[61427]: WARNING oslo_policy.policy [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 496.322025] env[61427]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 496.336202] env[61427]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Session token - acquiring [ 496.337001] env[61427]: DEBUG networking_nsxv3.common.locking [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61427) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 496.337220] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 496.337354] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 496.337714] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 496.346693] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 496.346932] env[61427]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 496.347088] env[61427]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61427) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 498.353319] env[61427]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Session token - acquiring [ 498.353735] env[61427]: DEBUG networking_nsxv3.common.locking [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61427) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 498.353816] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 498.353952] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 498.354164] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 498.363074] env[61427]: DEBUG oslo_concurrency.lockutils [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61427) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 498.363298] env[61427]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 498.363443] env[61427]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61427) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 500.364915] env[61427]: CRITICAL neutron [None req-dc5e964a-688f-407c-8bb6-fc6f704dd588 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 500.364915] env[61427]: ERROR neutron Traceback (most recent call last): [ 500.364915] env[61427]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 500.364915] env[61427]: ERROR neutron sys.exit(main()) [ 500.364915] env[61427]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 500.364915] env[61427]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 500.364915] env[61427]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 500.364915] env[61427]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 500.364915] env[61427]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 500.364915] env[61427]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 500.364915] env[61427]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 500.364915] env[61427]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 500.364915] env[61427]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 500.364915] env[61427]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 500.364915] env[61427]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 500.364915] env[61427]: ERROR neutron response = self.get(path=path, params=params) [ 500.366044] env[61427]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 500.366044] env[61427]: ERROR neutron raise RuntimeError(msg, last_err) [ 500.366044] env[61427]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 500.366044] env[61427]: ERROR neutron [ 503.335248] env[61642]: INFO neutron.common.config [-] Logging enabled! [ 503.335556] env[61642]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 503.335867] env[61642]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61642) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 503.336400] env[61642]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 503.336400] env[61642]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 503.337329] env[61642]: WARNING oslo_policy.policy [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 503.337693] env[61642]: WARNING oslo_policy.policy [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 503.339095] env[61642]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 503.353538] env[61642]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Session token - acquiring [ 503.354454] env[61642]: DEBUG networking_nsxv3.common.locking [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61642) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 503.354765] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 503.354994] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 503.355426] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 503.365010] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 503.365631] env[61642]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 503.365631] env[61642]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61642) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 505.371708] env[61642]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Session token - acquiring [ 505.372011] env[61642]: DEBUG networking_nsxv3.common.locking [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61642) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 505.372206] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 505.372327] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 505.372792] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 505.382302] env[61642]: DEBUG oslo_concurrency.lockutils [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61642) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 505.382599] env[61642]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 505.382745] env[61642]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61642) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 507.385139] env[61642]: CRITICAL neutron [None req-1e0a6cc2-946d-4e39-83c6-cc5d2e7310c4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 507.385139] env[61642]: ERROR neutron Traceback (most recent call last): [ 507.385139] env[61642]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 507.385139] env[61642]: ERROR neutron sys.exit(main()) [ 507.385139] env[61642]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 507.385139] env[61642]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 507.385139] env[61642]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 507.385139] env[61642]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 507.385139] env[61642]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 507.385139] env[61642]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 507.385139] env[61642]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 507.385139] env[61642]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 507.385139] env[61642]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 507.385139] env[61642]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 507.385139] env[61642]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 507.385139] env[61642]: ERROR neutron response = self.get(path=path, params=params) [ 507.385736] env[61642]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 507.385736] env[61642]: ERROR neutron raise RuntimeError(msg, last_err) [ 507.385736] env[61642]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 507.385736] env[61642]: ERROR neutron [ 510.118276] env[61824]: INFO neutron.common.config [-] Logging enabled! [ 510.118624] env[61824]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 510.118893] env[61824]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61824) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 510.119127] env[61824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 510.119215] env[61824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 510.120178] env[61824]: WARNING oslo_policy.policy [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 510.120405] env[61824]: WARNING oslo_policy.policy [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 510.122234] env[61824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 510.139629] env[61824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Session token - acquiring [ 510.140493] env[61824]: DEBUG networking_nsxv3.common.locking [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61824) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 510.140883] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 510.141067] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 510.141464] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 510.158089] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 510.158383] env[61824]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 510.158536] env[61824]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61824) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 512.164836] env[61824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Session token - acquiring [ 512.165145] env[61824]: DEBUG networking_nsxv3.common.locking [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61824) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 512.165306] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 512.165428] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 512.165715] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 512.175517] env[61824]: DEBUG oslo_concurrency.lockutils [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61824) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 512.175800] env[61824]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 512.175936] env[61824]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61824) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 514.180335] env[61824]: CRITICAL neutron [None req-f6b25341-2490-4b0b-b80f-950557e93e49 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 514.180335] env[61824]: ERROR neutron Traceback (most recent call last): [ 514.180335] env[61824]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 514.180335] env[61824]: ERROR neutron sys.exit(main()) [ 514.180335] env[61824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 514.180335] env[61824]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 514.180335] env[61824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 514.180335] env[61824]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 514.180335] env[61824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 514.180335] env[61824]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 514.180335] env[61824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 514.180335] env[61824]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 514.180335] env[61824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 514.180335] env[61824]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 514.180335] env[61824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 514.180335] env[61824]: ERROR neutron response = self.get(path=path, params=params) [ 514.181030] env[61824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 514.181030] env[61824]: ERROR neutron raise RuntimeError(msg, last_err) [ 514.181030] env[61824]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 514.181030] env[61824]: ERROR neutron [ 517.243329] env[62523]: INFO neutron.common.config [-] Logging enabled! [ 517.243597] env[62523]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 517.243909] env[62523]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62523) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 517.244240] env[62523]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 517.244366] env[62523]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 517.245372] env[62523]: WARNING oslo_policy.policy [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 517.245603] env[62523]: WARNING oslo_policy.policy [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 517.246940] env[62523]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 517.265352] env[62523]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Session token - acquiring [ 517.266226] env[62523]: DEBUG networking_nsxv3.common.locking [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62523) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 517.266438] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 517.266609] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 517.266951] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 517.280012] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 517.280412] env[62523]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 517.280412] env[62523]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62523) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 519.287802] env[62523]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Session token - acquiring [ 519.287802] env[62523]: DEBUG networking_nsxv3.common.locking [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62523) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 519.287802] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 519.288206] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 519.289615] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 519.300126] env[62523]: DEBUG oslo_concurrency.lockutils [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62523) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 519.300382] env[62523]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 519.300519] env[62523]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62523) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 521.303416] env[62523]: CRITICAL neutron [None req-7b36f67a-0644-4295-9ebd-f4c338f23a48 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 521.303416] env[62523]: ERROR neutron Traceback (most recent call last): [ 521.303416] env[62523]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 521.303416] env[62523]: ERROR neutron sys.exit(main()) [ 521.303416] env[62523]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 521.303416] env[62523]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 521.303416] env[62523]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 521.303416] env[62523]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 521.303416] env[62523]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 521.303416] env[62523]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 521.303416] env[62523]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 521.303416] env[62523]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 521.303416] env[62523]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 521.303416] env[62523]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 521.303416] env[62523]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 521.303416] env[62523]: ERROR neutron response = self.get(path=path, params=params) [ 521.304317] env[62523]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 521.304317] env[62523]: ERROR neutron raise RuntimeError(msg, last_err) [ 521.304317] env[62523]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 521.304317] env[62523]: ERROR neutron [ 524.084802] env[63124]: INFO neutron.common.config [-] Logging enabled! [ 524.085154] env[63124]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 524.085769] env[63124]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63124) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 524.086158] env[63124]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 524.086256] env[63124]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 524.087187] env[63124]: WARNING oslo_policy.policy [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 524.087401] env[63124]: WARNING oslo_policy.policy [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 524.088724] env[63124]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 524.103975] env[63124]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Session token - acquiring [ 524.104960] env[63124]: DEBUG networking_nsxv3.common.locking [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63124) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 524.105102] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 524.105221] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 524.105587] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 524.118205] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 524.118205] env[63124]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 524.118205] env[63124]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63124) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 526.127800] env[63124]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Session token - acquiring [ 526.127800] env[63124]: DEBUG networking_nsxv3.common.locking [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63124) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 526.127800] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 526.128213] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 526.128213] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 526.138263] env[63124]: DEBUG oslo_concurrency.lockutils [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63124) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 526.138541] env[63124]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 526.138658] env[63124]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63124) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 528.140938] env[63124]: CRITICAL neutron [None req-689b3c0f-9f34-4ecb-9e2d-4a39747e00c0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 528.140938] env[63124]: ERROR neutron Traceback (most recent call last): [ 528.140938] env[63124]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 528.140938] env[63124]: ERROR neutron sys.exit(main()) [ 528.140938] env[63124]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 528.140938] env[63124]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 528.140938] env[63124]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 528.140938] env[63124]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 528.140938] env[63124]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 528.140938] env[63124]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 528.140938] env[63124]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 528.140938] env[63124]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 528.140938] env[63124]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 528.140938] env[63124]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 528.140938] env[63124]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 528.140938] env[63124]: ERROR neutron response = self.get(path=path, params=params) [ 528.141734] env[63124]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 528.141734] env[63124]: ERROR neutron raise RuntimeError(msg, last_err) [ 528.141734] env[63124]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 528.141734] env[63124]: ERROR neutron [ 531.415201] env[63530]: INFO neutron.common.config [-] Logging enabled! [ 531.415476] env[63530]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 531.415721] env[63530]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63530) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 531.415943] env[63530]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 531.416063] env[63530]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 531.416948] env[63530]: WARNING oslo_policy.policy [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 531.417385] env[63530]: WARNING oslo_policy.policy [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 531.418646] env[63530]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 531.433289] env[63530]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Session token - acquiring [ 531.434145] env[63530]: DEBUG networking_nsxv3.common.locking [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63530) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 531.434359] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 531.434475] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 531.434857] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 531.444363] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 531.444599] env[63530]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 531.444747] env[63530]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63530) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 533.451131] env[63530]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Session token - acquiring [ 533.451563] env[63530]: DEBUG networking_nsxv3.common.locking [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63530) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 533.451761] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 533.451943] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 533.452260] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 533.465679] env[63530]: DEBUG oslo_concurrency.lockutils [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63530) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 533.465679] env[63530]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 533.465679] env[63530]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63530) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 535.468376] env[63530]: CRITICAL neutron [None req-45eeaa8a-04bb-4255-8eb7-6881bed580c8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 535.468376] env[63530]: ERROR neutron Traceback (most recent call last): [ 535.468376] env[63530]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 535.468376] env[63530]: ERROR neutron sys.exit(main()) [ 535.468376] env[63530]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 535.468376] env[63530]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 535.468376] env[63530]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 535.468376] env[63530]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 535.468376] env[63530]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 535.468376] env[63530]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 535.468376] env[63530]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 535.468376] env[63530]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 535.468376] env[63530]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 535.468376] env[63530]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 535.468376] env[63530]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 535.468376] env[63530]: ERROR neutron response = self.get(path=path, params=params) [ 535.469082] env[63530]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 535.469082] env[63530]: ERROR neutron raise RuntimeError(msg, last_err) [ 535.469082] env[63530]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 535.469082] env[63530]: ERROR neutron [ 538.478463] env[64069]: INFO neutron.common.config [-] Logging enabled! [ 538.478773] env[64069]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 538.479100] env[64069]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64069) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 538.479227] env[64069]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 538.479314] env[64069]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 538.480177] env[64069]: WARNING oslo_policy.policy [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 538.480388] env[64069]: WARNING oslo_policy.policy [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 538.481630] env[64069]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 538.502409] env[64069]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Session token - acquiring [ 538.506547] env[64069]: DEBUG networking_nsxv3.common.locking [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64069) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 538.506847] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 538.507054] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 538.508406] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 538.525029] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 538.526220] env[64069]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 538.526416] env[64069]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64069) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 540.533868] env[64069]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Session token - acquiring [ 540.534245] env[64069]: DEBUG networking_nsxv3.common.locking [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64069) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 540.534365] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 540.534469] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 540.534774] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 540.544007] env[64069]: DEBUG oslo_concurrency.lockutils [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64069) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 540.544263] env[64069]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 540.544396] env[64069]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64069) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 542.545502] env[64069]: CRITICAL neutron [None req-219b07ca-1048-4226-b4bd-d120c4409f96 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 542.545502] env[64069]: ERROR neutron Traceback (most recent call last): [ 542.545502] env[64069]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 542.545502] env[64069]: ERROR neutron sys.exit(main()) [ 542.545502] env[64069]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 542.545502] env[64069]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 542.545502] env[64069]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 542.545502] env[64069]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 542.545502] env[64069]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 542.545502] env[64069]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 542.545502] env[64069]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 542.545502] env[64069]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 542.545502] env[64069]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 542.545502] env[64069]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 542.545502] env[64069]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 542.545502] env[64069]: ERROR neutron response = self.get(path=path, params=params) [ 542.546228] env[64069]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 542.546228] env[64069]: ERROR neutron raise RuntimeError(msg, last_err) [ 542.546228] env[64069]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 542.546228] env[64069]: ERROR neutron [ 545.409007] env[64422]: INFO neutron.common.config [-] Logging enabled! [ 545.409240] env[64422]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 545.409599] env[64422]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64422) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 545.409803] env[64422]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 545.409892] env[64422]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 545.410755] env[64422]: WARNING oslo_policy.policy [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 545.410980] env[64422]: WARNING oslo_policy.policy [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 545.412350] env[64422]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 545.427118] env[64422]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Session token - acquiring [ 545.429842] env[64422]: DEBUG networking_nsxv3.common.locking [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64422) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 545.429842] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 545.430017] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 545.430017] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 545.439723] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 545.440012] env[64422]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 545.440190] env[64422]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64422) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 547.446374] env[64422]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Session token - acquiring [ 547.446701] env[64422]: DEBUG networking_nsxv3.common.locking [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64422) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 547.446853] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 547.446971] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 547.447241] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 547.456760] env[64422]: DEBUG oslo_concurrency.lockutils [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64422) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 547.457005] env[64422]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 547.457138] env[64422]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64422) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 549.459752] env[64422]: CRITICAL neutron [None req-6296f978-c1ed-470f-a273-68cc1e938c75 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 549.459752] env[64422]: ERROR neutron Traceback (most recent call last): [ 549.459752] env[64422]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 549.459752] env[64422]: ERROR neutron sys.exit(main()) [ 549.459752] env[64422]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 549.459752] env[64422]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 549.459752] env[64422]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 549.459752] env[64422]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 549.459752] env[64422]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 549.459752] env[64422]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 549.459752] env[64422]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 549.459752] env[64422]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 549.459752] env[64422]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 549.459752] env[64422]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 549.459752] env[64422]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 549.459752] env[64422]: ERROR neutron response = self.get(path=path, params=params) [ 549.460481] env[64422]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 549.460481] env[64422]: ERROR neutron raise RuntimeError(msg, last_err) [ 549.460481] env[64422]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 549.460481] env[64422]: ERROR neutron [ 552.623016] env[64863]: INFO neutron.common.config [-] Logging enabled! [ 552.623287] env[64863]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 552.623602] env[64863]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64863) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 552.623943] env[64863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 552.623943] env[64863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 552.624937] env[64863]: WARNING oslo_policy.policy [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 552.625077] env[64863]: WARNING oslo_policy.policy [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 552.626374] env[64863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 552.648759] env[64863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Session token - acquiring [ 552.648759] env[64863]: DEBUG networking_nsxv3.common.locking [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64863) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 552.648759] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 552.649339] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 552.649339] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 552.654517] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 552.654934] env[64863]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 552.655211] env[64863]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64863) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 554.662164] env[64863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Session token - acquiring [ 554.662164] env[64863]: DEBUG networking_nsxv3.common.locking [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64863) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 554.662459] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 554.662459] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 554.663017] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 554.672646] env[64863]: DEBUG oslo_concurrency.lockutils [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64863) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 554.672864] env[64863]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 554.673013] env[64863]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64863) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 556.675905] env[64863]: CRITICAL neutron [None req-b7657eac-31dd-4fc6-84aa-2062e8cb5217 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 556.675905] env[64863]: ERROR neutron Traceback (most recent call last): [ 556.675905] env[64863]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 556.675905] env[64863]: ERROR neutron sys.exit(main()) [ 556.675905] env[64863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 556.675905] env[64863]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 556.675905] env[64863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 556.675905] env[64863]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 556.675905] env[64863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 556.675905] env[64863]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 556.675905] env[64863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 556.675905] env[64863]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 556.675905] env[64863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 556.675905] env[64863]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 556.675905] env[64863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 556.675905] env[64863]: ERROR neutron response = self.get(path=path, params=params) [ 556.676784] env[64863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 556.676784] env[64863]: ERROR neutron raise RuntimeError(msg, last_err) [ 556.676784] env[64863]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 556.676784] env[64863]: ERROR neutron [ 559.614030] env[65293]: INFO neutron.common.config [-] Logging enabled! [ 559.614348] env[65293]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 559.614531] env[65293]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65293) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 559.614768] env[65293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 559.614877] env[65293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 559.615746] env[65293]: WARNING oslo_policy.policy [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 559.615966] env[65293]: WARNING oslo_policy.policy [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 559.617270] env[65293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 559.631430] env[65293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Session token - acquiring [ 559.632352] env[65293]: DEBUG networking_nsxv3.common.locking [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65293) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 559.632598] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 559.632723] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 559.633118] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 559.642941] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 559.644080] env[65293]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 559.644080] env[65293]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65293) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 561.650530] env[65293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Session token - acquiring [ 561.651412] env[65293]: DEBUG networking_nsxv3.common.locking [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65293) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 561.651412] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 561.651755] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 561.652830] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 561.665711] env[65293]: DEBUG oslo_concurrency.lockutils [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 561.666023] env[65293]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 561.666526] env[65293]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65293) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 563.670627] env[65293]: CRITICAL neutron [None req-95eb733d-1267-49b5-8aef-126deaf97f81 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 563.670627] env[65293]: ERROR neutron Traceback (most recent call last): [ 563.670627] env[65293]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 563.670627] env[65293]: ERROR neutron sys.exit(main()) [ 563.670627] env[65293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 563.670627] env[65293]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 563.670627] env[65293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 563.670627] env[65293]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 563.670627] env[65293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 563.670627] env[65293]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 563.670627] env[65293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 563.670627] env[65293]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 563.670627] env[65293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 563.670627] env[65293]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 563.670627] env[65293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 563.670627] env[65293]: ERROR neutron response = self.get(path=path, params=params) [ 563.671742] env[65293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 563.671742] env[65293]: ERROR neutron raise RuntimeError(msg, last_err) [ 563.671742] env[65293]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 563.671742] env[65293]: ERROR neutron [ 566.564530] env[65678]: INFO neutron.common.config [-] Logging enabled! [ 566.564530] env[65678]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 566.564530] env[65678]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65678) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 566.564530] env[65678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 566.564960] env[65678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 566.565697] env[65678]: WARNING oslo_policy.policy [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 566.565697] env[65678]: WARNING oslo_policy.policy [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 566.567051] env[65678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 566.614992] env[65678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Session token - acquiring [ 566.625813] env[65678]: DEBUG networking_nsxv3.common.locking [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65678) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 566.625813] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 566.626037] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 566.626283] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 566.636628] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 566.636911] env[65678]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 566.637218] env[65678]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65678) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 568.643340] env[65678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Session token - acquiring [ 568.643587] env[65678]: DEBUG networking_nsxv3.common.locking [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65678) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 568.643787] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 568.643898] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 568.644243] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 568.653559] env[65678]: DEBUG oslo_concurrency.lockutils [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 568.653818] env[65678]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 568.654123] env[65678]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65678) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 570.656906] env[65678]: CRITICAL neutron [None req-411123eb-90cf-4123-b5ce-71846c1223f6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 570.656906] env[65678]: ERROR neutron Traceback (most recent call last): [ 570.656906] env[65678]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 570.656906] env[65678]: ERROR neutron sys.exit(main()) [ 570.656906] env[65678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 570.656906] env[65678]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 570.656906] env[65678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 570.656906] env[65678]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 570.656906] env[65678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 570.656906] env[65678]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 570.656906] env[65678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 570.656906] env[65678]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 570.656906] env[65678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 570.656906] env[65678]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 570.656906] env[65678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 570.656906] env[65678]: ERROR neutron response = self.get(path=path, params=params) [ 570.657770] env[65678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 570.657770] env[65678]: ERROR neutron raise RuntimeError(msg, last_err) [ 570.657770] env[65678]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 570.657770] env[65678]: ERROR neutron [ 573.600532] env[65979]: INFO neutron.common.config [-] Logging enabled! [ 573.600884] env[65979]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 573.601069] env[65979]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65979) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 573.601294] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 573.601383] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 573.602318] env[65979]: WARNING oslo_policy.policy [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 573.602506] env[65979]: WARNING oslo_policy.policy [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 573.604045] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 573.621602] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Session token - acquiring [ 573.622519] env[65979]: DEBUG networking_nsxv3.common.locking [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65979) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 573.623323] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 573.623323] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 573.623563] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 573.633315] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 573.633574] env[65979]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 573.633748] env[65979]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65979) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 575.639833] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Session token - acquiring [ 575.640376] env[65979]: DEBUG networking_nsxv3.common.locking [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65979) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 575.640376] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 575.640483] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 575.640736] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 575.650093] env[65979]: DEBUG oslo_concurrency.lockutils [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 575.650368] env[65979]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 575.650889] env[65979]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65979) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 577.652911] env[65979]: CRITICAL neutron [None req-1ed92a14-46eb-48a6-a583-9223eaf1795d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 577.652911] env[65979]: ERROR neutron Traceback (most recent call last): [ 577.652911] env[65979]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 577.652911] env[65979]: ERROR neutron sys.exit(main()) [ 577.652911] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 577.652911] env[65979]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 577.652911] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 577.652911] env[65979]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 577.652911] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 577.652911] env[65979]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 577.652911] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 577.652911] env[65979]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 577.652911] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 577.652911] env[65979]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 577.652911] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 577.652911] env[65979]: ERROR neutron response = self.get(path=path, params=params) [ 577.653693] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 577.653693] env[65979]: ERROR neutron raise RuntimeError(msg, last_err) [ 577.653693] env[65979]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 577.653693] env[65979]: ERROR neutron [ 580.528516] env[66472]: INFO neutron.common.config [-] Logging enabled! [ 580.528516] env[66472]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 580.528516] env[66472]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66472) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 580.528516] env[66472]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 580.529142] env[66472]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 580.529756] env[66472]: WARNING oslo_policy.policy [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 580.529756] env[66472]: WARNING oslo_policy.policy [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 580.530983] env[66472]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 580.548466] env[66472]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Session token - acquiring [ 580.549322] env[66472]: DEBUG networking_nsxv3.common.locking [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66472) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 580.549550] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 580.549750] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 580.550292] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 580.561587] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 580.561929] env[66472]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 580.562082] env[66472]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66472) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 582.568388] env[66472]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Session token - acquiring [ 582.568751] env[66472]: DEBUG networking_nsxv3.common.locking [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66472) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 582.568947] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 582.569165] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 582.569347] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 582.578601] env[66472]: DEBUG oslo_concurrency.lockutils [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66472) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 582.578843] env[66472]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 582.578983] env[66472]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66472) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 584.581043] env[66472]: CRITICAL neutron [None req-e07446c6-a68a-461a-a9b3-142075124181 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 584.581043] env[66472]: ERROR neutron Traceback (most recent call last): [ 584.581043] env[66472]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 584.581043] env[66472]: ERROR neutron sys.exit(main()) [ 584.581043] env[66472]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 584.581043] env[66472]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 584.581043] env[66472]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 584.581043] env[66472]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 584.581043] env[66472]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 584.581043] env[66472]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 584.581043] env[66472]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 584.581043] env[66472]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 584.581043] env[66472]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 584.581043] env[66472]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 584.581043] env[66472]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 584.581043] env[66472]: ERROR neutron response = self.get(path=path, params=params) [ 584.581867] env[66472]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 584.581867] env[66472]: ERROR neutron raise RuntimeError(msg, last_err) [ 584.581867] env[66472]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 584.581867] env[66472]: ERROR neutron [ 587.795244] env[66876]: INFO neutron.common.config [-] Logging enabled! [ 587.795496] env[66876]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 587.795858] env[66876]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66876) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 587.795957] env[66876]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 587.796059] env[66876]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 587.796952] env[66876]: WARNING oslo_policy.policy [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 587.797167] env[66876]: WARNING oslo_policy.policy [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 587.798429] env[66876]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 587.813108] env[66876]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Session token - acquiring [ 587.814301] env[66876]: DEBUG networking_nsxv3.common.locking [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66876) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 587.814301] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 587.814447] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 587.814699] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 587.835626] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 587.835960] env[66876]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 587.836964] env[66876]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66876) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 589.843357] env[66876]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Session token - acquiring [ 589.843769] env[66876]: DEBUG networking_nsxv3.common.locking [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66876) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 589.843995] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 589.844135] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 589.844468] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 589.853576] env[66876]: DEBUG oslo_concurrency.lockutils [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66876) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 589.853817] env[66876]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 589.853984] env[66876]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66876) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 591.857064] env[66876]: CRITICAL neutron [None req-94f2309f-1977-4bd3-ac62-c2b8157a0755 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 591.857064] env[66876]: ERROR neutron Traceback (most recent call last): [ 591.857064] env[66876]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 591.857064] env[66876]: ERROR neutron sys.exit(main()) [ 591.857064] env[66876]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 591.857064] env[66876]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 591.857064] env[66876]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 591.857064] env[66876]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 591.857064] env[66876]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 591.857064] env[66876]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 591.857064] env[66876]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 591.857064] env[66876]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 591.857064] env[66876]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 591.857064] env[66876]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 591.857064] env[66876]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 591.857064] env[66876]: ERROR neutron response = self.get(path=path, params=params) [ 591.857670] env[66876]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 591.857670] env[66876]: ERROR neutron raise RuntimeError(msg, last_err) [ 591.857670] env[66876]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 591.857670] env[66876]: ERROR neutron [ 594.579853] env[67197]: INFO neutron.common.config [-] Logging enabled! [ 594.580176] env[67197]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 594.580435] env[67197]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67197) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 594.580671] env[67197]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 594.580777] env[67197]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 594.581713] env[67197]: WARNING oslo_policy.policy [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 594.581933] env[67197]: WARNING oslo_policy.policy [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 594.583282] env[67197]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 594.597407] env[67197]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Session token - acquiring [ 594.598295] env[67197]: DEBUG networking_nsxv3.common.locking [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67197) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 594.598522] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 594.598687] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 594.599018] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 594.609304] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 594.609717] env[67197]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 594.609957] env[67197]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67197) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 596.617629] env[67197]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Session token - acquiring [ 596.617629] env[67197]: DEBUG networking_nsxv3.common.locking [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67197) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 596.617629] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 596.618016] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 596.618016] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 596.626331] env[67197]: DEBUG oslo_concurrency.lockutils [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67197) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 596.626567] env[67197]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 596.626707] env[67197]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67197) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 598.629478] env[67197]: CRITICAL neutron [None req-d436e61a-552a-48e1-8a48-2013496d81f5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 598.629478] env[67197]: ERROR neutron Traceback (most recent call last): [ 598.629478] env[67197]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 598.629478] env[67197]: ERROR neutron sys.exit(main()) [ 598.629478] env[67197]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 598.629478] env[67197]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 598.629478] env[67197]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 598.629478] env[67197]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 598.629478] env[67197]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 598.629478] env[67197]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 598.629478] env[67197]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 598.629478] env[67197]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 598.629478] env[67197]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 598.629478] env[67197]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 598.629478] env[67197]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 598.629478] env[67197]: ERROR neutron response = self.get(path=path, params=params) [ 598.630173] env[67197]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 598.630173] env[67197]: ERROR neutron raise RuntimeError(msg, last_err) [ 598.630173] env[67197]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 598.630173] env[67197]: ERROR neutron [ 601.360199] env[67376]: INFO neutron.common.config [-] Logging enabled! [ 601.360522] env[67376]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 601.360716] env[67376]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67376) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 601.360935] env[67376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 601.361024] env[67376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 601.361883] env[67376]: WARNING oslo_policy.policy [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 601.362095] env[67376]: WARNING oslo_policy.policy [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 601.365235] env[67376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 601.379740] env[67376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Session token - acquiring [ 601.381440] env[67376]: DEBUG networking_nsxv3.common.locking [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67376) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 601.381759] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 601.381948] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 601.382404] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 601.401081] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 601.401081] env[67376]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 601.401081] env[67376]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67376) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 603.409381] env[67376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Session token - acquiring [ 603.409381] env[67376]: DEBUG networking_nsxv3.common.locking [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67376) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 603.409381] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 603.409843] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 603.409843] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 603.418657] env[67376]: DEBUG oslo_concurrency.lockutils [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 603.419119] env[67376]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 603.420117] env[67376]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67376) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 605.423220] env[67376]: CRITICAL neutron [None req-1a3cfe27-2786-40d3-9b4f-f949eadd09d4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 605.423220] env[67376]: ERROR neutron Traceback (most recent call last): [ 605.423220] env[67376]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 605.423220] env[67376]: ERROR neutron sys.exit(main()) [ 605.423220] env[67376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 605.423220] env[67376]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 605.423220] env[67376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 605.423220] env[67376]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 605.423220] env[67376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 605.423220] env[67376]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 605.423220] env[67376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 605.423220] env[67376]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 605.423220] env[67376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 605.423220] env[67376]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 605.423220] env[67376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 605.423220] env[67376]: ERROR neutron response = self.get(path=path, params=params) [ 605.423869] env[67376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 605.423869] env[67376]: ERROR neutron raise RuntimeError(msg, last_err) [ 605.423869] env[67376]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 605.423869] env[67376]: ERROR neutron [ 608.341334] env[67661]: INFO neutron.common.config [-] Logging enabled! [ 608.341599] env[67661]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 608.341839] env[67661]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67661) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 608.342777] env[67661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 608.342777] env[67661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 608.343080] env[67661]: WARNING oslo_policy.policy [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 608.343302] env[67661]: WARNING oslo_policy.policy [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 608.344552] env[67661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 608.363764] env[67661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Session token - acquiring [ 608.363764] env[67661]: DEBUG networking_nsxv3.common.locking [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67661) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 608.363764] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 608.363897] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 608.363897] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 608.373044] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 608.373310] env[67661]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 608.373455] env[67661]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67661) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 610.379544] env[67661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Session token - acquiring [ 610.380016] env[67661]: DEBUG networking_nsxv3.common.locking [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67661) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 610.380088] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 610.380191] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 610.380472] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 610.389730] env[67661]: DEBUG oslo_concurrency.lockutils [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 610.389968] env[67661]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 610.390110] env[67661]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67661) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 612.392925] env[67661]: CRITICAL neutron [None req-e8b7ea6a-367b-420f-a2ce-aa68a271c9e5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 612.392925] env[67661]: ERROR neutron Traceback (most recent call last): [ 612.392925] env[67661]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 612.392925] env[67661]: ERROR neutron sys.exit(main()) [ 612.392925] env[67661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 612.392925] env[67661]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 612.392925] env[67661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 612.392925] env[67661]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 612.392925] env[67661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 612.392925] env[67661]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 612.392925] env[67661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 612.392925] env[67661]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 612.392925] env[67661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 612.392925] env[67661]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 612.392925] env[67661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 612.392925] env[67661]: ERROR neutron response = self.get(path=path, params=params) [ 612.393627] env[67661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 612.393627] env[67661]: ERROR neutron raise RuntimeError(msg, last_err) [ 612.393627] env[67661]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 612.393627] env[67661]: ERROR neutron [ 615.146323] env[67915]: INFO neutron.common.config [-] Logging enabled! [ 615.146555] env[67915]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 615.146870] env[67915]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67915) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 615.147100] env[67915]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 615.147197] env[67915]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 615.149049] env[67915]: WARNING oslo_policy.policy [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 615.149049] env[67915]: WARNING oslo_policy.policy [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 615.149670] env[67915]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 615.165051] env[67915]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Session token - acquiring [ 615.166026] env[67915]: DEBUG networking_nsxv3.common.locking [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67915) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 615.166234] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 615.166428] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 615.166831] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 615.176855] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 615.177123] env[67915]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 615.177301] env[67915]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67915) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 617.183588] env[67915]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Session token - acquiring [ 617.184021] env[67915]: DEBUG networking_nsxv3.common.locking [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67915) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 617.184224] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 617.184348] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 617.184632] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 617.195805] env[67915]: DEBUG oslo_concurrency.lockutils [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67915) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 617.195996] env[67915]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 617.196159] env[67915]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67915) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 619.199259] env[67915]: CRITICAL neutron [None req-631cb17e-befa-40b4-9423-200d00e799cd None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 619.199259] env[67915]: ERROR neutron Traceback (most recent call last): [ 619.199259] env[67915]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 619.199259] env[67915]: ERROR neutron sys.exit(main()) [ 619.199259] env[67915]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 619.199259] env[67915]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 619.199259] env[67915]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 619.199259] env[67915]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 619.199259] env[67915]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 619.199259] env[67915]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 619.199259] env[67915]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 619.199259] env[67915]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 619.199259] env[67915]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 619.199259] env[67915]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 619.199259] env[67915]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 619.199259] env[67915]: ERROR neutron response = self.get(path=path, params=params) [ 619.199976] env[67915]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 619.199976] env[67915]: ERROR neutron raise RuntimeError(msg, last_err) [ 619.199976] env[67915]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 619.199976] env[67915]: ERROR neutron [ 622.251621] env[68236]: INFO neutron.common.config [-] Logging enabled! [ 622.251979] env[68236]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 622.252192] env[68236]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68236) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 622.252419] env[68236]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 622.252512] env[68236]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 622.253385] env[68236]: WARNING oslo_policy.policy [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 622.253608] env[68236]: WARNING oslo_policy.policy [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 622.255896] env[68236]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 622.270659] env[68236]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Session token - acquiring [ 622.271502] env[68236]: DEBUG networking_nsxv3.common.locking [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68236) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 622.271735] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 622.271865] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 622.272263] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 622.288182] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 622.288479] env[68236]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 622.288646] env[68236]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68236) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 624.294824] env[68236]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Session token - acquiring [ 624.295209] env[68236]: DEBUG networking_nsxv3.common.locking [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68236) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 624.295421] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 624.295548] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 624.295812] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 624.304947] env[68236]: DEBUG oslo_concurrency.lockutils [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68236) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 624.305194] env[68236]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 624.305330] env[68236]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68236) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 626.312256] env[68236]: CRITICAL neutron [None req-8562e315-cb60-46e4-8d2a-72e849539c37 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 626.312256] env[68236]: ERROR neutron Traceback (most recent call last): [ 626.312256] env[68236]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 626.312256] env[68236]: ERROR neutron sys.exit(main()) [ 626.312256] env[68236]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 626.312256] env[68236]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 626.312256] env[68236]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 626.312256] env[68236]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 626.312256] env[68236]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 626.312256] env[68236]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 626.312256] env[68236]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 626.312256] env[68236]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 626.312256] env[68236]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 626.312256] env[68236]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 626.312256] env[68236]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 626.312256] env[68236]: ERROR neutron response = self.get(path=path, params=params) [ 626.312915] env[68236]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 626.312915] env[68236]: ERROR neutron raise RuntimeError(msg, last_err) [ 626.312915] env[68236]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 626.312915] env[68236]: ERROR neutron [ 629.172382] env[68371]: INFO neutron.common.config [-] Logging enabled! [ 629.172620] env[68371]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 629.172930] env[68371]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68371) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 629.173158] env[68371]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 629.173350] env[68371]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 629.174242] env[68371]: WARNING oslo_policy.policy [None req-be502899-ac4b-415e-9151-a4428b704981 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 629.174547] env[68371]: WARNING oslo_policy.policy [None req-be502899-ac4b-415e-9151-a4428b704981 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 629.175836] env[68371]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 629.190538] env[68371]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Session token - acquiring [ 629.191369] env[68371]: DEBUG networking_nsxv3.common.locking [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68371) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 629.191588] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 629.191726] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 629.192161] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 629.201736] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 629.201996] env[68371]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 629.202141] env[68371]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68371) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 631.208535] env[68371]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Session token - acquiring [ 631.208885] env[68371]: DEBUG networking_nsxv3.common.locking [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68371) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 631.209063] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 631.209189] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 631.209522] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 631.218466] env[68371]: DEBUG oslo_concurrency.lockutils [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68371) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 631.218730] env[68371]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 631.218870] env[68371]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68371) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 633.220896] env[68371]: CRITICAL neutron [None req-be502899-ac4b-415e-9151-a4428b704981 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 633.220896] env[68371]: ERROR neutron Traceback (most recent call last): [ 633.220896] env[68371]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 633.220896] env[68371]: ERROR neutron sys.exit(main()) [ 633.220896] env[68371]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 633.220896] env[68371]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 633.220896] env[68371]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 633.220896] env[68371]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 633.220896] env[68371]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 633.220896] env[68371]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 633.220896] env[68371]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 633.220896] env[68371]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 633.220896] env[68371]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 633.220896] env[68371]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 633.220896] env[68371]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 633.220896] env[68371]: ERROR neutron response = self.get(path=path, params=params) [ 633.221558] env[68371]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 633.221558] env[68371]: ERROR neutron raise RuntimeError(msg, last_err) [ 633.221558] env[68371]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 633.221558] env[68371]: ERROR neutron [ 635.949009] env[68655]: INFO neutron.common.config [-] Logging enabled! [ 635.949313] env[68655]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 635.949556] env[68655]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68655) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 635.949764] env[68655]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 635.949859] env[68655]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 635.950717] env[68655]: WARNING oslo_policy.policy [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 635.950947] env[68655]: WARNING oslo_policy.policy [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 635.952216] env[68655]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 635.966487] env[68655]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Session token - acquiring [ 635.967294] env[68655]: DEBUG networking_nsxv3.common.locking [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68655) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 635.967506] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 635.967638] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 635.968040] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 635.984976] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 635.985288] env[68655]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 635.985440] env[68655]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68655) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 637.991886] env[68655]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Session token - acquiring [ 637.992326] env[68655]: DEBUG networking_nsxv3.common.locking [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68655) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 637.992526] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 637.992710] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 637.992993] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 638.002736] env[68655]: DEBUG oslo_concurrency.lockutils [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68655) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 638.002783] env[68655]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 638.003240] env[68655]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68655) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 640.004899] env[68655]: CRITICAL neutron [None req-041603a8-5f03-4cee-856d-0ee6f17f3214 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 640.004899] env[68655]: ERROR neutron Traceback (most recent call last): [ 640.004899] env[68655]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 640.004899] env[68655]: ERROR neutron sys.exit(main()) [ 640.004899] env[68655]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 640.004899] env[68655]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 640.004899] env[68655]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 640.004899] env[68655]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 640.004899] env[68655]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 640.004899] env[68655]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 640.004899] env[68655]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 640.004899] env[68655]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 640.004899] env[68655]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 640.004899] env[68655]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 640.004899] env[68655]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 640.004899] env[68655]: ERROR neutron response = self.get(path=path, params=params) [ 640.005500] env[68655]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 640.005500] env[68655]: ERROR neutron raise RuntimeError(msg, last_err) [ 640.005500] env[68655]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 640.005500] env[68655]: ERROR neutron [ 642.757642] env[68759]: INFO neutron.common.config [-] Logging enabled! [ 642.758000] env[68759]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 642.758266] env[68759]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68759) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 642.758496] env[68759]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 642.758590] env[68759]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 642.759408] env[68759]: WARNING oslo_policy.policy [None req-8e42db56-b605-4086-afea-71306f4551ca None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 642.759636] env[68759]: WARNING oslo_policy.policy [None req-8e42db56-b605-4086-afea-71306f4551ca None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 642.760918] env[68759]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 642.774960] env[68759]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Session token - acquiring [ 642.775753] env[68759]: DEBUG networking_nsxv3.common.locking [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68759) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 642.775951] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 642.776093] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 642.776454] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 642.785483] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 642.785765] env[68759]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 642.785914] env[68759]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68759) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 644.791948] env[68759]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Session token - acquiring [ 644.792363] env[68759]: DEBUG networking_nsxv3.common.locking [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68759) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 644.792574] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 644.792715] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 644.792980] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 644.804790] env[68759]: DEBUG oslo_concurrency.lockutils [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68759) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 644.805028] env[68759]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 644.805169] env[68759]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68759) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 646.807453] env[68759]: CRITICAL neutron [None req-8e42db56-b605-4086-afea-71306f4551ca None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 646.807453] env[68759]: ERROR neutron Traceback (most recent call last): [ 646.807453] env[68759]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 646.807453] env[68759]: ERROR neutron sys.exit(main()) [ 646.807453] env[68759]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 646.807453] env[68759]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 646.807453] env[68759]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 646.807453] env[68759]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 646.807453] env[68759]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 646.807453] env[68759]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 646.807453] env[68759]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 646.807453] env[68759]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 646.807453] env[68759]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 646.807453] env[68759]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 646.807453] env[68759]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 646.807453] env[68759]: ERROR neutron response = self.get(path=path, params=params) [ 646.808191] env[68759]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 646.808191] env[68759]: ERROR neutron raise RuntimeError(msg, last_err) [ 646.808191] env[68759]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 646.808191] env[68759]: ERROR neutron [ 649.554601] env[68826]: INFO neutron.common.config [-] Logging enabled! [ 649.554899] env[68826]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 649.555140] env[68826]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68826) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 649.555385] env[68826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 649.555530] env[68826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 649.556413] env[68826]: WARNING oslo_policy.policy [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 649.556640] env[68826]: WARNING oslo_policy.policy [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 649.557954] env[68826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 649.572124] env[68826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Session token - acquiring [ 649.572944] env[68826]: DEBUG networking_nsxv3.common.locking [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68826) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 649.573152] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 649.573279] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 649.573644] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 649.582574] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 649.582818] env[68826]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 649.582962] env[68826]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68826) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 651.590872] env[68826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Session token - acquiring [ 651.591756] env[68826]: DEBUG networking_nsxv3.common.locking [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68826) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 651.591756] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 651.591910] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 651.591910] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 651.602639] env[68826]: DEBUG oslo_concurrency.lockutils [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 651.602971] env[68826]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 651.603142] env[68826]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68826) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 653.604867] env[68826]: CRITICAL neutron [None req-1bf0d68f-a841-43f9-a562-7b8bb2b80da4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 653.604867] env[68826]: ERROR neutron Traceback (most recent call last): [ 653.604867] env[68826]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 653.604867] env[68826]: ERROR neutron sys.exit(main()) [ 653.604867] env[68826]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 653.604867] env[68826]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 653.604867] env[68826]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 653.604867] env[68826]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 653.604867] env[68826]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 653.604867] env[68826]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 653.604867] env[68826]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 653.604867] env[68826]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 653.604867] env[68826]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 653.604867] env[68826]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 653.604867] env[68826]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 653.604867] env[68826]: ERROR neutron response = self.get(path=path, params=params) [ 653.605575] env[68826]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 653.605575] env[68826]: ERROR neutron raise RuntimeError(msg, last_err) [ 653.605575] env[68826]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 653.605575] env[68826]: ERROR neutron [ 656.385677] env[68896]: INFO neutron.common.config [-] Logging enabled! [ 656.386029] env[68896]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 656.386470] env[68896]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68896) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 656.386470] env[68896]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 656.386570] env[68896]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 656.387434] env[68896]: WARNING oslo_policy.policy [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 656.387646] env[68896]: WARNING oslo_policy.policy [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 656.388924] env[68896]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 656.403954] env[68896]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Session token - acquiring [ 656.404939] env[68896]: DEBUG networking_nsxv3.common.locking [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68896) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 656.405515] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 656.405515] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 656.405665] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 656.415404] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 656.415680] env[68896]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 656.415831] env[68896]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68896) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 658.421853] env[68896]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Session token - acquiring [ 658.422149] env[68896]: DEBUG networking_nsxv3.common.locking [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68896) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 658.422354] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 658.422482] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 658.422756] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 658.431523] env[68896]: DEBUG oslo_concurrency.lockutils [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68896) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 658.431887] env[68896]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 658.431887] env[68896]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68896) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 660.432962] env[68896]: CRITICAL neutron [None req-dcdf32ef-74b4-47b5-88bc-c21cf1b69422 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 660.432962] env[68896]: ERROR neutron Traceback (most recent call last): [ 660.432962] env[68896]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 660.432962] env[68896]: ERROR neutron sys.exit(main()) [ 660.432962] env[68896]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 660.432962] env[68896]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 660.432962] env[68896]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 660.432962] env[68896]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 660.432962] env[68896]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 660.432962] env[68896]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 660.432962] env[68896]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 660.432962] env[68896]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 660.432962] env[68896]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 660.432962] env[68896]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 660.432962] env[68896]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 660.432962] env[68896]: ERROR neutron response = self.get(path=path, params=params) [ 660.433708] env[68896]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 660.433708] env[68896]: ERROR neutron raise RuntimeError(msg, last_err) [ 660.433708] env[68896]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 660.433708] env[68896]: ERROR neutron [ 663.261203] env[69009]: INFO neutron.common.config [-] Logging enabled! [ 663.261452] env[69009]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 663.261786] env[69009]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69009) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 663.262004] env[69009]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 663.262095] env[69009]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 663.263008] env[69009]: WARNING oslo_policy.policy [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 663.263206] env[69009]: WARNING oslo_policy.policy [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 663.264567] env[69009]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 663.279440] env[69009]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Session token - acquiring [ 663.280787] env[69009]: DEBUG networking_nsxv3.common.locking [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69009) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 663.280787] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 663.280970] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 663.281182] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 663.293721] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 663.293721] env[69009]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 663.293721] env[69009]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69009) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 665.298597] env[69009]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Session token - acquiring [ 665.298877] env[69009]: DEBUG networking_nsxv3.common.locking [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69009) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 665.299027] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 665.299507] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 665.299507] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 665.309072] env[69009]: DEBUG oslo_concurrency.lockutils [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69009) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 665.309318] env[69009]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 665.309450] env[69009]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69009) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 667.312900] env[69009]: CRITICAL neutron [None req-b1dd7674-0a45-4a77-9ba8-86f21ec6c489 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 667.312900] env[69009]: ERROR neutron Traceback (most recent call last): [ 667.312900] env[69009]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 667.312900] env[69009]: ERROR neutron sys.exit(main()) [ 667.312900] env[69009]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 667.312900] env[69009]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 667.312900] env[69009]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 667.312900] env[69009]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 667.312900] env[69009]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 667.312900] env[69009]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 667.312900] env[69009]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 667.312900] env[69009]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 667.312900] env[69009]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 667.312900] env[69009]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 667.312900] env[69009]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 667.312900] env[69009]: ERROR neutron response = self.get(path=path, params=params) [ 667.313667] env[69009]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 667.313667] env[69009]: ERROR neutron raise RuntimeError(msg, last_err) [ 667.313667] env[69009]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 667.313667] env[69009]: ERROR neutron [ 670.032551] env[69324]: INFO neutron.common.config [-] Logging enabled! [ 670.032822] env[69324]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 670.033126] env[69324]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69324) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 670.033399] env[69324]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 670.033493] env[69324]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 670.034352] env[69324]: WARNING oslo_policy.policy [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 670.034561] env[69324]: WARNING oslo_policy.policy [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 670.035903] env[69324]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 670.053529] env[69324]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Session token - acquiring [ 670.054382] env[69324]: DEBUG networking_nsxv3.common.locking [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69324) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 670.054581] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 670.054702] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 670.055134] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 670.064911] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 670.065186] env[69324]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 670.065335] env[69324]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69324) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 672.071398] env[69324]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Session token - acquiring [ 672.071795] env[69324]: DEBUG networking_nsxv3.common.locking [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69324) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 672.071992] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 672.072131] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 672.072400] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 672.081385] env[69324]: DEBUG oslo_concurrency.lockutils [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69324) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 672.081673] env[69324]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 672.081814] env[69324]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69324) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 674.084774] env[69324]: CRITICAL neutron [None req-c2d1b3a4-2f34-4b60-a065-dbb778499aaa None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 674.084774] env[69324]: ERROR neutron Traceback (most recent call last): [ 674.084774] env[69324]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 674.084774] env[69324]: ERROR neutron sys.exit(main()) [ 674.084774] env[69324]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 674.084774] env[69324]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 674.084774] env[69324]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 674.084774] env[69324]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 674.084774] env[69324]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 674.084774] env[69324]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 674.084774] env[69324]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 674.084774] env[69324]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 674.084774] env[69324]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 674.084774] env[69324]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 674.084774] env[69324]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 674.084774] env[69324]: ERROR neutron response = self.get(path=path, params=params) [ 674.085526] env[69324]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 674.085526] env[69324]: ERROR neutron raise RuntimeError(msg, last_err) [ 674.085526] env[69324]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 674.085526] env[69324]: ERROR neutron [ 676.706411] env[69527]: INFO neutron.common.config [-] Logging enabled! [ 676.706411] env[69527]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 676.706761] env[69527]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69527) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 676.706917] env[69527]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 676.707006] env[69527]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 676.707870] env[69527]: WARNING oslo_policy.policy [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 676.708109] env[69527]: WARNING oslo_policy.policy [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 676.709350] env[69527]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 676.723472] env[69527]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Session token - acquiring [ 676.724277] env[69527]: DEBUG networking_nsxv3.common.locking [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69527) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 676.724477] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 676.724601] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 676.724963] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 676.733983] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 676.734222] env[69527]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 676.734367] env[69527]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69527) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 678.740562] env[69527]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Session token - acquiring [ 678.741203] env[69527]: DEBUG networking_nsxv3.common.locking [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69527) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 678.741419] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 678.741885] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 678.741885] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 678.751354] env[69527]: DEBUG oslo_concurrency.lockutils [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69527) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 678.751616] env[69527]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 678.751761] env[69527]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69527) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 680.753398] env[69527]: CRITICAL neutron [None req-0d66f670-aeca-44ce-ae05-7a57acd2434d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 680.753398] env[69527]: ERROR neutron Traceback (most recent call last): [ 680.753398] env[69527]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 680.753398] env[69527]: ERROR neutron sys.exit(main()) [ 680.753398] env[69527]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 680.753398] env[69527]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 680.753398] env[69527]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 680.753398] env[69527]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 680.753398] env[69527]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 680.753398] env[69527]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 680.753398] env[69527]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 680.753398] env[69527]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 680.753398] env[69527]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 680.753398] env[69527]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 680.753398] env[69527]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 680.753398] env[69527]: ERROR neutron response = self.get(path=path, params=params) [ 680.754186] env[69527]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 680.754186] env[69527]: ERROR neutron raise RuntimeError(msg, last_err) [ 680.754186] env[69527]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 680.754186] env[69527]: ERROR neutron [ 683.690547] env[69714]: INFO neutron.common.config [-] Logging enabled! [ 683.690880] env[69714]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 683.691059] env[69714]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69714) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 683.691300] env[69714]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 683.691391] env[69714]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 683.692270] env[69714]: WARNING oslo_policy.policy [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 683.692479] env[69714]: WARNING oslo_policy.policy [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 683.700516] env[69714]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 683.716413] env[69714]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Session token - acquiring [ 683.717369] env[69714]: DEBUG networking_nsxv3.common.locking [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69714) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 683.717603] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 683.718057] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 683.718192] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 683.727892] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 683.728239] env[69714]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 683.728410] env[69714]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69714) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 685.736827] env[69714]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Session token - acquiring [ 685.737232] env[69714]: DEBUG networking_nsxv3.common.locking [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69714) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 685.737455] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 685.737649] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 685.737984] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 685.749789] env[69714]: DEBUG oslo_concurrency.lockutils [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69714) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 685.749789] env[69714]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 685.749789] env[69714]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69714) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 687.751377] env[69714]: CRITICAL neutron [None req-f95da061-989a-4be4-bef4-832d7ffe2991 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 687.751377] env[69714]: ERROR neutron Traceback (most recent call last): [ 687.751377] env[69714]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 687.751377] env[69714]: ERROR neutron sys.exit(main()) [ 687.751377] env[69714]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 687.751377] env[69714]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 687.751377] env[69714]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 687.751377] env[69714]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 687.751377] env[69714]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 687.751377] env[69714]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 687.751377] env[69714]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 687.751377] env[69714]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 687.751377] env[69714]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 687.751377] env[69714]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 687.751377] env[69714]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 687.751377] env[69714]: ERROR neutron response = self.get(path=path, params=params) [ 687.752231] env[69714]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 687.752231] env[69714]: ERROR neutron raise RuntimeError(msg, last_err) [ 687.752231] env[69714]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 687.752231] env[69714]: ERROR neutron [ 690.548136] env[70001]: INFO neutron.common.config [-] Logging enabled! [ 690.548136] env[70001]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 690.548460] env[70001]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70001) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 690.548516] env[70001]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 690.548618] env[70001]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 690.549516] env[70001]: WARNING oslo_policy.policy [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 690.549778] env[70001]: WARNING oslo_policy.policy [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 690.551234] env[70001]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 690.565305] env[70001]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Session token - acquiring [ 690.566156] env[70001]: DEBUG networking_nsxv3.common.locking [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70001) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 690.566354] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 690.566459] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 690.571085] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 690.577937] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 690.578241] env[70001]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 690.578398] env[70001]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70001) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 692.584782] env[70001]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Session token - acquiring [ 692.584782] env[70001]: DEBUG networking_nsxv3.common.locking [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70001) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 692.585126] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 692.585126] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 692.585311] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 692.594807] env[70001]: DEBUG oslo_concurrency.lockutils [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70001) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 692.595062] env[70001]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 692.595202] env[70001]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70001) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 694.596878] env[70001]: CRITICAL neutron [None req-d21c6651-4165-4b46-99af-b43292aa5494 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 694.596878] env[70001]: ERROR neutron Traceback (most recent call last): [ 694.596878] env[70001]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 694.596878] env[70001]: ERROR neutron sys.exit(main()) [ 694.596878] env[70001]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 694.596878] env[70001]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 694.596878] env[70001]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 694.596878] env[70001]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 694.596878] env[70001]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 694.596878] env[70001]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 694.596878] env[70001]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 694.596878] env[70001]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 694.596878] env[70001]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 694.596878] env[70001]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 694.596878] env[70001]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 694.596878] env[70001]: ERROR neutron response = self.get(path=path, params=params) [ 694.597707] env[70001]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 694.597707] env[70001]: ERROR neutron raise RuntimeError(msg, last_err) [ 694.597707] env[70001]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 694.597707] env[70001]: ERROR neutron [ 697.382178] env[70265]: INFO neutron.common.config [-] Logging enabled! [ 697.382443] env[70265]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 697.383197] env[70265]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70265) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 697.383434] env[70265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 697.383533] env[70265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 697.384425] env[70265]: WARNING oslo_policy.policy [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 697.384655] env[70265]: WARNING oslo_policy.policy [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 697.386218] env[70265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 697.402656] env[70265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Session token - acquiring [ 697.403972] env[70265]: DEBUG networking_nsxv3.common.locking [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70265) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 697.404355] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 697.404506] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 697.405187] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 697.416257] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 697.416515] env[70265]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 697.416656] env[70265]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70265) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 699.424971] env[70265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Session token - acquiring [ 699.425315] env[70265]: DEBUG networking_nsxv3.common.locking [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70265) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 699.425535] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 699.425667] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 699.425939] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 699.435413] env[70265]: DEBUG oslo_concurrency.lockutils [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 699.435650] env[70265]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 699.435791] env[70265]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70265) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 701.436921] env[70265]: CRITICAL neutron [None req-baa04e59-a316-4753-9853-e4de9adc2130 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 701.436921] env[70265]: ERROR neutron Traceback (most recent call last): [ 701.436921] env[70265]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 701.436921] env[70265]: ERROR neutron sys.exit(main()) [ 701.436921] env[70265]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 701.436921] env[70265]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 701.436921] env[70265]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 701.436921] env[70265]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 701.436921] env[70265]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 701.436921] env[70265]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 701.436921] env[70265]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 701.436921] env[70265]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 701.436921] env[70265]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 701.436921] env[70265]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 701.436921] env[70265]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 701.436921] env[70265]: ERROR neutron response = self.get(path=path, params=params) [ 701.437693] env[70265]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 701.437693] env[70265]: ERROR neutron raise RuntimeError(msg, last_err) [ 701.437693] env[70265]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 701.437693] env[70265]: ERROR neutron [ 703.984555] env[70521]: INFO neutron.common.config [-] Logging enabled! [ 703.984875] env[70521]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 703.985141] env[70521]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70521) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 703.985385] env[70521]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 703.985479] env[70521]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 703.986737] env[70521]: WARNING oslo_policy.policy [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 703.986954] env[70521]: WARNING oslo_policy.policy [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 703.988548] env[70521]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 704.004929] env[70521]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Session token - acquiring [ 704.005799] env[70521]: DEBUG networking_nsxv3.common.locking [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70521) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 704.005996] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 704.006121] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 704.006492] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 704.017548] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 704.017548] env[70521]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 704.017548] env[70521]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70521) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 706.023145] env[70521]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Session token - acquiring [ 706.023498] env[70521]: DEBUG networking_nsxv3.common.locking [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70521) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 706.023735] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 706.025345] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 706.025345] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 706.035977] env[70521]: DEBUG oslo_concurrency.lockutils [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70521) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 706.036407] env[70521]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 706.036642] env[70521]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70521) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 708.039538] env[70521]: CRITICAL neutron [None req-eecf7873-2015-4ecf-9b74-bc1b0db6aba1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 708.039538] env[70521]: ERROR neutron Traceback (most recent call last): [ 708.039538] env[70521]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 708.039538] env[70521]: ERROR neutron sys.exit(main()) [ 708.039538] env[70521]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 708.039538] env[70521]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 708.039538] env[70521]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 708.039538] env[70521]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 708.039538] env[70521]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 708.039538] env[70521]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 708.039538] env[70521]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 708.039538] env[70521]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 708.039538] env[70521]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 708.039538] env[70521]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 708.039538] env[70521]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 708.039538] env[70521]: ERROR neutron response = self.get(path=path, params=params) [ 708.040360] env[70521]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 708.040360] env[70521]: ERROR neutron raise RuntimeError(msg, last_err) [ 708.040360] env[70521]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 708.040360] env[70521]: ERROR neutron [ 710.890945] env[70649]: INFO neutron.common.config [-] Logging enabled! [ 710.891264] env[70649]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 710.891664] env[70649]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70649) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 710.891838] env[70649]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 710.891932] env[70649]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 710.892786] env[70649]: WARNING oslo_policy.policy [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 710.892994] env[70649]: WARNING oslo_policy.policy [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 710.894198] env[70649]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 710.908920] env[70649]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Session token - acquiring [ 710.909728] env[70649]: DEBUG networking_nsxv3.common.locking [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70649) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 710.909936] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 710.910069] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 710.910435] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 710.919712] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 710.919975] env[70649]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 710.920137] env[70649]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70649) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 712.926273] env[70649]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Session token - acquiring [ 712.926660] env[70649]: DEBUG networking_nsxv3.common.locking [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70649) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 712.927772] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 712.927772] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 712.927772] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 712.943196] env[70649]: DEBUG oslo_concurrency.lockutils [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70649) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 712.943497] env[70649]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 712.943659] env[70649]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70649) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 714.945020] env[70649]: CRITICAL neutron [None req-9c9d3c7d-ba4d-4293-b605-38c0a13f3fa2 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 714.945020] env[70649]: ERROR neutron Traceback (most recent call last): [ 714.945020] env[70649]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 714.945020] env[70649]: ERROR neutron sys.exit(main()) [ 714.945020] env[70649]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 714.945020] env[70649]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 714.945020] env[70649]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 714.945020] env[70649]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 714.945020] env[70649]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 714.945020] env[70649]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 714.945020] env[70649]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 714.945020] env[70649]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 714.945020] env[70649]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 714.945020] env[70649]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 714.945020] env[70649]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 714.945020] env[70649]: ERROR neutron response = self.get(path=path, params=params) [ 714.945819] env[70649]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 714.945819] env[70649]: ERROR neutron raise RuntimeError(msg, last_err) [ 714.945819] env[70649]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 714.945819] env[70649]: ERROR neutron [ 717.827082] env[70783]: INFO neutron.common.config [-] Logging enabled! [ 717.827377] env[70783]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 717.827633] env[70783]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70783) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 717.827870] env[70783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 717.827965] env[70783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 717.828840] env[70783]: WARNING oslo_policy.policy [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 717.829055] env[70783]: WARNING oslo_policy.policy [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 717.830306] env[70783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 717.845360] env[70783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Session token - acquiring [ 717.848028] env[70783]: DEBUG networking_nsxv3.common.locking [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70783) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 717.848028] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 717.848343] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 717.848343] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 717.856247] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 717.856343] env[70783]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 717.856444] env[70783]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70783) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 719.862387] env[70783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Session token - acquiring [ 719.862832] env[70783]: DEBUG networking_nsxv3.common.locking [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70783) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 719.863059] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 719.863179] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 719.863446] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 719.872787] env[70783]: DEBUG oslo_concurrency.lockutils [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 719.873039] env[70783]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 719.873177] env[70783]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70783) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 721.877848] env[70783]: CRITICAL neutron [None req-03c5f7b4-fc38-46bd-939e-a14f1b718a17 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 721.877848] env[70783]: ERROR neutron Traceback (most recent call last): [ 721.877848] env[70783]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 721.877848] env[70783]: ERROR neutron sys.exit(main()) [ 721.877848] env[70783]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 721.877848] env[70783]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 721.877848] env[70783]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 721.877848] env[70783]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 721.877848] env[70783]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 721.877848] env[70783]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 721.877848] env[70783]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 721.877848] env[70783]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 721.877848] env[70783]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 721.877848] env[70783]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 721.877848] env[70783]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 721.877848] env[70783]: ERROR neutron response = self.get(path=path, params=params) [ 721.878668] env[70783]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 721.878668] env[70783]: ERROR neutron raise RuntimeError(msg, last_err) [ 721.878668] env[70783]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 721.878668] env[70783]: ERROR neutron [ 724.730843] env[71143]: INFO neutron.common.config [-] Logging enabled! [ 724.731174] env[71143]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 724.731468] env[71143]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71143) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 724.731734] env[71143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 724.731843] env[71143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 724.732821] env[71143]: WARNING oslo_policy.policy [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 724.733013] env[71143]: WARNING oslo_policy.policy [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 724.734266] env[71143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 724.750153] env[71143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Session token - acquiring [ 724.751001] env[71143]: DEBUG networking_nsxv3.common.locking [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71143) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 724.751212] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 724.751340] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 724.751735] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 724.760992] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 724.761245] env[71143]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 724.761387] env[71143]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71143) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 726.774618] env[71143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Session token - acquiring [ 726.774618] env[71143]: DEBUG networking_nsxv3.common.locking [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71143) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 726.774618] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 726.774945] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 726.774945] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 726.783851] env[71143]: DEBUG oslo_concurrency.lockutils [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 726.784363] env[71143]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 726.784664] env[71143]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71143) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 728.787965] env[71143]: CRITICAL neutron [None req-c5a47e60-4def-4797-957b-13a48cd8fc33 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 728.787965] env[71143]: ERROR neutron Traceback (most recent call last): [ 728.787965] env[71143]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 728.787965] env[71143]: ERROR neutron sys.exit(main()) [ 728.787965] env[71143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 728.787965] env[71143]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 728.787965] env[71143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 728.787965] env[71143]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 728.787965] env[71143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 728.787965] env[71143]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 728.787965] env[71143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 728.787965] env[71143]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 728.787965] env[71143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 728.787965] env[71143]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 728.787965] env[71143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 728.787965] env[71143]: ERROR neutron response = self.get(path=path, params=params) [ 728.788989] env[71143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 728.788989] env[71143]: ERROR neutron raise RuntimeError(msg, last_err) [ 728.788989] env[71143]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 728.788989] env[71143]: ERROR neutron [ 731.467617] env[71276]: INFO neutron.common.config [-] Logging enabled! [ 731.467987] env[71276]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 731.468159] env[71276]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71276) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 731.468380] env[71276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 731.468468] env[71276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 731.469308] env[71276]: WARNING oslo_policy.policy [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 731.469522] env[71276]: WARNING oslo_policy.policy [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 731.470713] env[71276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 731.485327] env[71276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Session token - acquiring [ 731.486658] env[71276]: DEBUG networking_nsxv3.common.locking [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71276) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 731.486992] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 731.487224] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 731.487764] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 731.497584] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 731.498012] env[71276]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 731.498301] env[71276]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71276) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 733.504790] env[71276]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Session token - acquiring [ 733.505101] env[71276]: DEBUG networking_nsxv3.common.locking [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71276) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 733.505101] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 733.505210] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 733.505487] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 733.514585] env[71276]: DEBUG oslo_concurrency.lockutils [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71276) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 733.514835] env[71276]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 733.514975] env[71276]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71276) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 735.516958] env[71276]: CRITICAL neutron [None req-dab149ef-3c57-4430-9772-e4414f7525f6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 735.516958] env[71276]: ERROR neutron Traceback (most recent call last): [ 735.516958] env[71276]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 735.516958] env[71276]: ERROR neutron sys.exit(main()) [ 735.516958] env[71276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 735.516958] env[71276]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 735.516958] env[71276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 735.516958] env[71276]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 735.516958] env[71276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 735.516958] env[71276]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 735.516958] env[71276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 735.516958] env[71276]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 735.516958] env[71276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 735.516958] env[71276]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 735.516958] env[71276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 735.516958] env[71276]: ERROR neutron response = self.get(path=path, params=params) [ 735.517834] env[71276]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 735.517834] env[71276]: ERROR neutron raise RuntimeError(msg, last_err) [ 735.517834] env[71276]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 735.517834] env[71276]: ERROR neutron [ 738.381174] env[71421]: INFO neutron.common.config [-] Logging enabled! [ 738.381484] env[71421]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 738.381783] env[71421]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71421) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 738.382216] env[71421]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 738.382216] env[71421]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 738.382955] env[71421]: WARNING oslo_policy.policy [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 738.383280] env[71421]: WARNING oslo_policy.policy [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 738.384429] env[71421]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 738.399663] env[71421]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Session token - acquiring [ 738.400561] env[71421]: DEBUG networking_nsxv3.common.locking [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71421) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 738.400778] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 738.400900] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 738.401274] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 738.410543] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 738.410881] env[71421]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 738.410937] env[71421]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71421) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 740.417226] env[71421]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Session token - acquiring [ 740.417583] env[71421]: DEBUG networking_nsxv3.common.locking [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71421) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 740.417780] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 740.417905] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 740.418184] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 740.428665] env[71421]: DEBUG oslo_concurrency.lockutils [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71421) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 740.428935] env[71421]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 740.429077] env[71421]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71421) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 742.432051] env[71421]: CRITICAL neutron [None req-2b6f217e-e82d-4942-a13c-773710536e14 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 742.432051] env[71421]: ERROR neutron Traceback (most recent call last): [ 742.432051] env[71421]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 742.432051] env[71421]: ERROR neutron sys.exit(main()) [ 742.432051] env[71421]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 742.432051] env[71421]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 742.432051] env[71421]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 742.432051] env[71421]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 742.432051] env[71421]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 742.432051] env[71421]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 742.432051] env[71421]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 742.432051] env[71421]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 742.432051] env[71421]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 742.432051] env[71421]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 742.432051] env[71421]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 742.432051] env[71421]: ERROR neutron response = self.get(path=path, params=params) [ 742.432913] env[71421]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 742.432913] env[71421]: ERROR neutron raise RuntimeError(msg, last_err) [ 742.432913] env[71421]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 742.432913] env[71421]: ERROR neutron [ 745.047125] env[71639]: INFO neutron.common.config [-] Logging enabled! [ 745.047693] env[71639]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 745.048579] env[71639]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71639) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 745.048873] env[71639]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 745.048932] env[71639]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 745.049796] env[71639]: WARNING oslo_policy.policy [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 745.050023] env[71639]: WARNING oslo_policy.policy [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 745.051264] env[71639]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 745.068592] env[71639]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Session token - acquiring [ 745.069590] env[71639]: DEBUG networking_nsxv3.common.locking [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71639) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 745.069815] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 745.069946] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 745.070368] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 745.080499] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 745.080773] env[71639]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 745.080925] env[71639]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71639) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 747.087052] env[71639]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Session token - acquiring [ 747.087343] env[71639]: DEBUG networking_nsxv3.common.locking [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71639) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 747.087534] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 747.087665] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 747.087943] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 747.097341] env[71639]: DEBUG oslo_concurrency.lockutils [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71639) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 747.097583] env[71639]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 747.097748] env[71639]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71639) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 749.100675] env[71639]: CRITICAL neutron [None req-43110c38-107c-4970-bed0-f008b0579ea1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 749.100675] env[71639]: ERROR neutron Traceback (most recent call last): [ 749.100675] env[71639]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 749.100675] env[71639]: ERROR neutron sys.exit(main()) [ 749.100675] env[71639]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 749.100675] env[71639]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 749.100675] env[71639]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 749.100675] env[71639]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 749.100675] env[71639]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 749.100675] env[71639]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 749.100675] env[71639]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 749.100675] env[71639]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 749.100675] env[71639]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 749.100675] env[71639]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 749.100675] env[71639]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 749.100675] env[71639]: ERROR neutron response = self.get(path=path, params=params) [ 749.102196] env[71639]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 749.102196] env[71639]: ERROR neutron raise RuntimeError(msg, last_err) [ 749.102196] env[71639]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 749.102196] env[71639]: ERROR neutron [ 751.764340] env[71777]: INFO neutron.common.config [-] Logging enabled! [ 751.764772] env[71777]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 751.764915] env[71777]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71777) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 751.765151] env[71777]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 751.765235] env[71777]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 751.766107] env[71777]: WARNING oslo_policy.policy [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 751.766327] env[71777]: WARNING oslo_policy.policy [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 751.767558] env[71777]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 751.783277] env[71777]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Session token - acquiring [ 751.784160] env[71777]: DEBUG networking_nsxv3.common.locking [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71777) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 751.784377] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 751.784505] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 751.784879] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 751.794922] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 751.795159] env[71777]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 751.795307] env[71777]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71777) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 753.801953] env[71777]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Session token - acquiring [ 753.802250] env[71777]: DEBUG networking_nsxv3.common.locking [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71777) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 753.802440] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 753.802566] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 753.802887] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 753.811885] env[71777]: DEBUG oslo_concurrency.lockutils [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71777) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 753.812149] env[71777]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 753.817657] env[71777]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71777) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 755.824061] env[71777]: CRITICAL neutron [None req-4f3582b7-97f5-4003-9285-942a1d051b3d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 755.824061] env[71777]: ERROR neutron Traceback (most recent call last): [ 755.824061] env[71777]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 755.824061] env[71777]: ERROR neutron sys.exit(main()) [ 755.824061] env[71777]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 755.824061] env[71777]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 755.824061] env[71777]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 755.824061] env[71777]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 755.824061] env[71777]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 755.824061] env[71777]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 755.824061] env[71777]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 755.824061] env[71777]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 755.824061] env[71777]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 755.824061] env[71777]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 755.824061] env[71777]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 755.824061] env[71777]: ERROR neutron response = self.get(path=path, params=params) [ 755.824767] env[71777]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 755.824767] env[71777]: ERROR neutron raise RuntimeError(msg, last_err) [ 755.824767] env[71777]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 755.824767] env[71777]: ERROR neutron [ 758.521636] env[71883]: INFO neutron.common.config [-] Logging enabled! [ 758.521853] env[71883]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 758.522193] env[71883]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71883) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 758.522414] env[71883]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 758.522506] env[71883]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 758.523378] env[71883]: WARNING oslo_policy.policy [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 758.523598] env[71883]: WARNING oslo_policy.policy [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 758.524898] env[71883]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 758.544722] env[71883]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Session token - acquiring [ 758.544722] env[71883]: DEBUG networking_nsxv3.common.locking [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71883) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 758.544722] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 758.545079] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 758.545079] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 758.554890] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 758.554890] env[71883]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 758.554890] env[71883]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71883) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 760.558495] env[71883]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Session token - acquiring [ 760.558746] env[71883]: DEBUG networking_nsxv3.common.locking [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71883) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 760.558824] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 760.558896] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 760.559269] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 760.568439] env[71883]: DEBUG oslo_concurrency.lockutils [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71883) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 760.568682] env[71883]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 760.568816] env[71883]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71883) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 762.571892] env[71883]: CRITICAL neutron [None req-e1bb15a5-166f-4eba-a67a-e0b0d3447567 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 762.571892] env[71883]: ERROR neutron Traceback (most recent call last): [ 762.571892] env[71883]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 762.571892] env[71883]: ERROR neutron sys.exit(main()) [ 762.571892] env[71883]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 762.571892] env[71883]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 762.571892] env[71883]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 762.571892] env[71883]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 762.571892] env[71883]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 762.571892] env[71883]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 762.571892] env[71883]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 762.571892] env[71883]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 762.571892] env[71883]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 762.571892] env[71883]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 762.571892] env[71883]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 762.571892] env[71883]: ERROR neutron response = self.get(path=path, params=params) [ 762.572539] env[71883]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 762.572539] env[71883]: ERROR neutron raise RuntimeError(msg, last_err) [ 762.572539] env[71883]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 762.572539] env[71883]: ERROR neutron [ 765.385014] env[72132]: INFO neutron.common.config [-] Logging enabled! [ 765.385329] env[72132]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 765.385542] env[72132]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72132) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 765.385770] env[72132]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 765.385940] env[72132]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 765.386772] env[72132]: WARNING oslo_policy.policy [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 765.386985] env[72132]: WARNING oslo_policy.policy [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 765.388210] env[72132]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 765.402638] env[72132]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Session token - acquiring [ 765.403550] env[72132]: DEBUG networking_nsxv3.common.locking [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72132) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 765.403764] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 765.403926] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 765.404374] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 765.414261] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 765.414536] env[72132]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 765.414703] env[72132]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72132) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 767.421392] env[72132]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Session token - acquiring [ 767.424516] env[72132]: DEBUG networking_nsxv3.common.locking [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72132) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 767.424516] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 767.424715] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 767.424715] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 767.433110] env[72132]: DEBUG oslo_concurrency.lockutils [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72132) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 767.433110] env[72132]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 767.433110] env[72132]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72132) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 769.435899] env[72132]: CRITICAL neutron [None req-fd920f66-57f5-4f6e-93ed-93f928e1cbae None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 769.435899] env[72132]: ERROR neutron Traceback (most recent call last): [ 769.435899] env[72132]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 769.435899] env[72132]: ERROR neutron sys.exit(main()) [ 769.435899] env[72132]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 769.435899] env[72132]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 769.435899] env[72132]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 769.435899] env[72132]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 769.435899] env[72132]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 769.435899] env[72132]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 769.435899] env[72132]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 769.435899] env[72132]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 769.435899] env[72132]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 769.435899] env[72132]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 769.435899] env[72132]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 769.435899] env[72132]: ERROR neutron response = self.get(path=path, params=params) [ 769.436606] env[72132]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 769.436606] env[72132]: ERROR neutron raise RuntimeError(msg, last_err) [ 769.436606] env[72132]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 769.436606] env[72132]: ERROR neutron [ 772.275639] env[72380]: INFO neutron.common.config [-] Logging enabled! [ 772.275890] env[72380]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 772.276187] env[72380]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72380) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 772.276417] env[72380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 772.276520] env[72380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 772.277347] env[72380]: WARNING oslo_policy.policy [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 772.277572] env[72380]: WARNING oslo_policy.policy [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 772.278762] env[72380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 772.292939] env[72380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Session token - acquiring [ 772.293697] env[72380]: DEBUG networking_nsxv3.common.locking [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72380) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 772.293894] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 772.294018] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 772.294388] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 772.303530] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 772.303772] env[72380]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 772.303913] env[72380]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72380) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 774.310044] env[72380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Session token - acquiring [ 774.310354] env[72380]: DEBUG networking_nsxv3.common.locking [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72380) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 774.311162] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 774.311162] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 774.311162] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 774.320477] env[72380]: DEBUG oslo_concurrency.lockutils [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 774.320727] env[72380]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 774.320862] env[72380]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72380) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 776.323726] env[72380]: CRITICAL neutron [None req-67b02910-bfe9-4df2-89ca-8ab27fd11386 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 776.323726] env[72380]: ERROR neutron Traceback (most recent call last): [ 776.323726] env[72380]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 776.323726] env[72380]: ERROR neutron sys.exit(main()) [ 776.323726] env[72380]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 776.323726] env[72380]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 776.323726] env[72380]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 776.323726] env[72380]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 776.323726] env[72380]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 776.323726] env[72380]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 776.323726] env[72380]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 776.323726] env[72380]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 776.323726] env[72380]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 776.323726] env[72380]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 776.323726] env[72380]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 776.323726] env[72380]: ERROR neutron response = self.get(path=path, params=params) [ 776.324515] env[72380]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 776.324515] env[72380]: ERROR neutron raise RuntimeError(msg, last_err) [ 776.324515] env[72380]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 776.324515] env[72380]: ERROR neutron [ 779.064737] env[72522]: INFO neutron.common.config [-] Logging enabled! [ 779.065109] env[72522]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 779.065388] env[72522]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72522) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 779.065943] env[72522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 779.065943] env[72522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 779.066704] env[72522]: WARNING oslo_policy.policy [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 779.066969] env[72522]: WARNING oslo_policy.policy [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 779.068253] env[72522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 779.083758] env[72522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Session token - acquiring [ 779.084628] env[72522]: DEBUG networking_nsxv3.common.locking [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72522) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 779.084840] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 779.084971] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 779.085358] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 779.095055] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 779.095344] env[72522]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 779.095529] env[72522]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72522) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 781.101921] env[72522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Session token - acquiring [ 781.102266] env[72522]: DEBUG networking_nsxv3.common.locking [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72522) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 781.102568] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 781.102774] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 781.103121] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 781.122960] env[72522]: DEBUG oslo_concurrency.lockutils [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72522) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 781.122960] env[72522]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 781.122960] env[72522]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72522) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 783.120920] env[72522]: CRITICAL neutron [None req-439ca85c-7214-49af-b039-24a48f6aa4e7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 783.120920] env[72522]: ERROR neutron Traceback (most recent call last): [ 783.120920] env[72522]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 783.120920] env[72522]: ERROR neutron sys.exit(main()) [ 783.120920] env[72522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 783.120920] env[72522]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 783.120920] env[72522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 783.120920] env[72522]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 783.120920] env[72522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 783.120920] env[72522]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 783.120920] env[72522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 783.120920] env[72522]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 783.120920] env[72522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 783.120920] env[72522]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 783.120920] env[72522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 783.120920] env[72522]: ERROR neutron response = self.get(path=path, params=params) [ 783.121561] env[72522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 783.121561] env[72522]: ERROR neutron raise RuntimeError(msg, last_err) [ 783.121561] env[72522]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 783.121561] env[72522]: ERROR neutron [ 785.682298] env[72641]: INFO neutron.common.config [-] Logging enabled! [ 785.682298] env[72641]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 785.682751] env[72641]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72641) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 785.682796] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 785.682885] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 785.683717] env[72641]: WARNING oslo_policy.policy [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 785.683938] env[72641]: WARNING oslo_policy.policy [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 785.685145] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 785.699479] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Session token - acquiring [ 785.700330] env[72641]: DEBUG networking_nsxv3.common.locking [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72641) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 785.700579] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 785.700745] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 785.701148] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 785.710269] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 785.710569] env[72641]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 785.710754] env[72641]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72641) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 787.717441] env[72641]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Session token - acquiring [ 787.717441] env[72641]: DEBUG networking_nsxv3.common.locking [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72641) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 787.717441] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 787.717744] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 787.717876] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 787.726961] env[72641]: DEBUG oslo_concurrency.lockutils [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72641) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 787.727237] env[72641]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 787.727379] env[72641]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72641) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 789.732047] env[72641]: CRITICAL neutron [None req-d5e1a1ba-aaa6-48e4-8ff2-8506bf737a6d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 789.732047] env[72641]: ERROR neutron Traceback (most recent call last): [ 789.732047] env[72641]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 789.732047] env[72641]: ERROR neutron sys.exit(main()) [ 789.732047] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 789.732047] env[72641]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 789.732047] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 789.732047] env[72641]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 789.732047] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 789.732047] env[72641]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 789.732047] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 789.732047] env[72641]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 789.732047] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 789.732047] env[72641]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 789.732047] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 789.732047] env[72641]: ERROR neutron response = self.get(path=path, params=params) [ 789.732868] env[72641]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 789.732868] env[72641]: ERROR neutron raise RuntimeError(msg, last_err) [ 789.732868] env[72641]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 789.732868] env[72641]: ERROR neutron [ 792.623367] env[72744]: INFO neutron.common.config [-] Logging enabled! [ 792.623668] env[72744]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 792.623870] env[72744]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72744) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 792.624127] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 792.624222] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 792.625039] env[72744]: WARNING oslo_policy.policy [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 792.625261] env[72744]: WARNING oslo_policy.policy [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 792.626435] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 792.640650] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Session token - acquiring [ 792.641460] env[72744]: DEBUG networking_nsxv3.common.locking [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72744) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 792.641611] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 792.641769] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 792.642147] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 792.651313] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 792.651581] env[72744]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 792.651733] env[72744]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72744) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 794.657778] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Session token - acquiring [ 794.658092] env[72744]: DEBUG networking_nsxv3.common.locking [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72744) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 794.658242] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 794.658364] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 794.658643] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 794.668455] env[72744]: DEBUG oslo_concurrency.lockutils [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 794.669219] env[72744]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 794.669219] env[72744]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72744) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 796.672029] env[72744]: CRITICAL neutron [None req-8d9b9018-19c5-4b79-a83c-01da7058db44 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 796.672029] env[72744]: ERROR neutron Traceback (most recent call last): [ 796.672029] env[72744]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 796.672029] env[72744]: ERROR neutron sys.exit(main()) [ 796.672029] env[72744]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 796.672029] env[72744]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 796.672029] env[72744]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 796.672029] env[72744]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 796.672029] env[72744]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 796.672029] env[72744]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 796.672029] env[72744]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 796.672029] env[72744]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 796.672029] env[72744]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 796.672029] env[72744]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 796.672029] env[72744]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 796.672029] env[72744]: ERROR neutron response = self.get(path=path, params=params) [ 796.672707] env[72744]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 796.672707] env[72744]: ERROR neutron raise RuntimeError(msg, last_err) [ 796.672707] env[72744]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 796.672707] env[72744]: ERROR neutron [ 799.259341] env[73034]: INFO neutron.common.config [-] Logging enabled! [ 799.259654] env[73034]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 799.259931] env[73034]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73034) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 799.260194] env[73034]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 799.260293] env[73034]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 799.261179] env[73034]: WARNING oslo_policy.policy [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 799.261402] env[73034]: WARNING oslo_policy.policy [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 799.262659] env[73034]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 799.283070] env[73034]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Session token - acquiring [ 799.283920] env[73034]: DEBUG networking_nsxv3.common.locking [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73034) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 799.284151] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 799.284289] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 799.284682] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 799.295498] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 799.295498] env[73034]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 799.295498] env[73034]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73034) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 801.302899] env[73034]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Session token - acquiring [ 801.302899] env[73034]: DEBUG networking_nsxv3.common.locking [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73034) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 801.302899] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 801.303299] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 801.303299] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 801.312578] env[73034]: DEBUG oslo_concurrency.lockutils [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73034) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 801.312797] env[73034]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 801.312963] env[73034]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73034) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 803.315871] env[73034]: CRITICAL neutron [None req-b8052877-087b-4e1d-bafa-4829d79fab2e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 803.315871] env[73034]: ERROR neutron Traceback (most recent call last): [ 803.315871] env[73034]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 803.315871] env[73034]: ERROR neutron sys.exit(main()) [ 803.315871] env[73034]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 803.315871] env[73034]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 803.315871] env[73034]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 803.315871] env[73034]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 803.315871] env[73034]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 803.315871] env[73034]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 803.315871] env[73034]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 803.315871] env[73034]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 803.315871] env[73034]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 803.315871] env[73034]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 803.315871] env[73034]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 803.315871] env[73034]: ERROR neutron response = self.get(path=path, params=params) [ 803.316623] env[73034]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 803.316623] env[73034]: ERROR neutron raise RuntimeError(msg, last_err) [ 803.316623] env[73034]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 803.316623] env[73034]: ERROR neutron [ 805.983538] env[73142]: INFO neutron.common.config [-] Logging enabled! [ 805.983957] env[73142]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 805.984416] env[73142]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73142) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 805.984660] env[73142]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 805.984756] env[73142]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 805.985701] env[73142]: WARNING oslo_policy.policy [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 805.985934] env[73142]: WARNING oslo_policy.policy [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 805.987320] env[73142]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 806.006488] env[73142]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Session token - acquiring [ 806.007474] env[73142]: DEBUG networking_nsxv3.common.locking [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73142) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 806.007609] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 806.007706] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 806.008109] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 806.017690] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 806.018010] env[73142]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 806.019592] env[73142]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73142) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 808.024213] env[73142]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Session token - acquiring [ 808.024695] env[73142]: DEBUG networking_nsxv3.common.locking [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73142) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 808.024695] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 808.024868] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 808.025112] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 808.033910] env[73142]: DEBUG oslo_concurrency.lockutils [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73142) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 808.034146] env[73142]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 808.034283] env[73142]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73142) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 810.040333] env[73142]: CRITICAL neutron [None req-be5d47bb-d410-46f9-bb90-65ecad0c16ec None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 810.040333] env[73142]: ERROR neutron Traceback (most recent call last): [ 810.040333] env[73142]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 810.040333] env[73142]: ERROR neutron sys.exit(main()) [ 810.040333] env[73142]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 810.040333] env[73142]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 810.040333] env[73142]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 810.040333] env[73142]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 810.040333] env[73142]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 810.040333] env[73142]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 810.040333] env[73142]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 810.040333] env[73142]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 810.040333] env[73142]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 810.040333] env[73142]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 810.040333] env[73142]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 810.040333] env[73142]: ERROR neutron response = self.get(path=path, params=params) [ 810.041114] env[73142]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 810.041114] env[73142]: ERROR neutron raise RuntimeError(msg, last_err) [ 810.041114] env[73142]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 810.041114] env[73142]: ERROR neutron [ 812.803205] env[73184]: INFO neutron.common.config [-] Logging enabled! [ 812.803541] env[73184]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 812.803786] env[73184]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73184) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 812.804032] env[73184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 812.804132] env[73184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 812.804984] env[73184]: WARNING oslo_policy.policy [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 812.805251] env[73184]: WARNING oslo_policy.policy [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 812.806526] env[73184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 812.821204] env[73184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Session token - acquiring [ 812.822067] env[73184]: DEBUG networking_nsxv3.common.locking [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73184) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 812.822262] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 812.822387] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 812.822789] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 812.834979] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 812.834979] env[73184]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 812.834979] env[73184]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73184) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 814.845359] env[73184]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Session token - acquiring [ 814.845670] env[73184]: DEBUG networking_nsxv3.common.locking [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73184) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 814.845840] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 814.845960] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 814.846243] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 814.856454] env[73184]: DEBUG oslo_concurrency.lockutils [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73184) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 814.856772] env[73184]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 814.856980] env[73184]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73184) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 816.859895] env[73184]: CRITICAL neutron [None req-e05ee088-336c-44d3-b53a-d0fdf6f8c7d3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 816.859895] env[73184]: ERROR neutron Traceback (most recent call last): [ 816.859895] env[73184]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 816.859895] env[73184]: ERROR neutron sys.exit(main()) [ 816.859895] env[73184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 816.859895] env[73184]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 816.859895] env[73184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 816.859895] env[73184]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 816.859895] env[73184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 816.859895] env[73184]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 816.859895] env[73184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 816.859895] env[73184]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 816.859895] env[73184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 816.859895] env[73184]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 816.859895] env[73184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 816.859895] env[73184]: ERROR neutron response = self.get(path=path, params=params) [ 816.860638] env[73184]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 816.860638] env[73184]: ERROR neutron raise RuntimeError(msg, last_err) [ 816.860638] env[73184]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 816.860638] env[73184]: ERROR neutron [ 819.442083] env[73426]: INFO neutron.common.config [-] Logging enabled! [ 819.442343] env[73426]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 819.442582] env[73426]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73426) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 819.442880] env[73426]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 819.442942] env[73426]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 819.443703] env[73426]: WARNING oslo_policy.policy [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 819.443937] env[73426]: WARNING oslo_policy.policy [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 819.445157] env[73426]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 819.459053] env[73426]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Session token - acquiring [ 819.459806] env[73426]: DEBUG networking_nsxv3.common.locking [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73426) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 819.460029] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 819.460159] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 819.460517] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 819.469332] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 819.469564] env[73426]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 819.469721] env[73426]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73426) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 821.477744] env[73426]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Session token - acquiring [ 821.477744] env[73426]: DEBUG networking_nsxv3.common.locking [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73426) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 821.477744] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 821.478147] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 821.478147] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 821.486231] env[73426]: DEBUG oslo_concurrency.lockutils [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73426) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 821.486587] env[73426]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 821.486830] env[73426]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73426) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 823.489192] env[73426]: CRITICAL neutron [None req-42a9f07c-ac9e-4d7d-a165-e62b0cbdb1b1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 823.489192] env[73426]: ERROR neutron Traceback (most recent call last): [ 823.489192] env[73426]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 823.489192] env[73426]: ERROR neutron sys.exit(main()) [ 823.489192] env[73426]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 823.489192] env[73426]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 823.489192] env[73426]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 823.489192] env[73426]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 823.489192] env[73426]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 823.489192] env[73426]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 823.489192] env[73426]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 823.489192] env[73426]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 823.489192] env[73426]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 823.489192] env[73426]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 823.489192] env[73426]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 823.489192] env[73426]: ERROR neutron response = self.get(path=path, params=params) [ 823.489882] env[73426]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 823.489882] env[73426]: ERROR neutron raise RuntimeError(msg, last_err) [ 823.489882] env[73426]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 823.489882] env[73426]: ERROR neutron [ 826.237409] env[73554]: INFO neutron.common.config [-] Logging enabled! [ 826.237717] env[73554]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 826.237863] env[73554]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73554) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 826.238096] env[73554]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 826.238189] env[73554]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 826.239086] env[73554]: WARNING oslo_policy.policy [None req-7a05083c-a295-472d-a8f8-49554945412b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 826.239315] env[73554]: WARNING oslo_policy.policy [None req-7a05083c-a295-472d-a8f8-49554945412b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 826.240648] env[73554]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 826.254925] env[73554]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Session token - acquiring [ 826.255790] env[73554]: DEBUG networking_nsxv3.common.locking [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73554) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 826.255991] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 826.256157] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 826.256524] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 826.269443] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 826.269770] env[73554]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 826.269903] env[73554]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73554) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 828.275889] env[73554]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Session token - acquiring [ 828.276265] env[73554]: DEBUG networking_nsxv3.common.locking [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73554) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 828.276433] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 828.276559] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 828.276843] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 828.286302] env[73554]: DEBUG oslo_concurrency.lockutils [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73554) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 828.286555] env[73554]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 828.286695] env[73554]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73554) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 830.288942] env[73554]: CRITICAL neutron [None req-7a05083c-a295-472d-a8f8-49554945412b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 830.288942] env[73554]: ERROR neutron Traceback (most recent call last): [ 830.288942] env[73554]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 830.288942] env[73554]: ERROR neutron sys.exit(main()) [ 830.288942] env[73554]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 830.288942] env[73554]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 830.288942] env[73554]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 830.288942] env[73554]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 830.288942] env[73554]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 830.288942] env[73554]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 830.288942] env[73554]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 830.288942] env[73554]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 830.288942] env[73554]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 830.288942] env[73554]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 830.288942] env[73554]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 830.288942] env[73554]: ERROR neutron response = self.get(path=path, params=params) [ 830.289558] env[73554]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 830.289558] env[73554]: ERROR neutron raise RuntimeError(msg, last_err) [ 830.289558] env[73554]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 830.289558] env[73554]: ERROR neutron [ 833.052609] env[73779]: INFO neutron.common.config [-] Logging enabled! [ 833.052981] env[73779]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 833.053189] env[73779]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73779) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 833.053422] env[73779]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 833.053515] env[73779]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 833.054434] env[73779]: WARNING oslo_policy.policy [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 833.054593] env[73779]: WARNING oslo_policy.policy [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 833.055905] env[73779]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 833.074429] env[73779]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Session token - acquiring [ 833.075544] env[73779]: DEBUG networking_nsxv3.common.locking [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73779) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 833.076102] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 833.076373] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 833.077110] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 833.087878] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 833.088156] env[73779]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 833.088301] env[73779]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73779) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 835.094977] env[73779]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Session token - acquiring [ 835.095407] env[73779]: DEBUG networking_nsxv3.common.locking [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73779) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 835.095603] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 835.095767] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 835.096044] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 835.105077] env[73779]: DEBUG oslo_concurrency.lockutils [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73779) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 835.105324] env[73779]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 835.105463] env[73779]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73779) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 837.108386] env[73779]: CRITICAL neutron [None req-af1bb5d0-e7b4-420a-ac8e-53ba6f05e821 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 837.108386] env[73779]: ERROR neutron Traceback (most recent call last): [ 837.108386] env[73779]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 837.108386] env[73779]: ERROR neutron sys.exit(main()) [ 837.108386] env[73779]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 837.108386] env[73779]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 837.108386] env[73779]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 837.108386] env[73779]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 837.108386] env[73779]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 837.108386] env[73779]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 837.108386] env[73779]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 837.108386] env[73779]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 837.108386] env[73779]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 837.108386] env[73779]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 837.108386] env[73779]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 837.108386] env[73779]: ERROR neutron response = self.get(path=path, params=params) [ 837.109106] env[73779]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 837.109106] env[73779]: ERROR neutron raise RuntimeError(msg, last_err) [ 837.109106] env[73779]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 837.109106] env[73779]: ERROR neutron [ 839.881224] env[73934]: INFO neutron.common.config [-] Logging enabled! [ 839.881509] env[73934]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 839.882045] env[73934]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73934) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 839.882295] env[73934]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 839.882384] env[73934]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 839.883843] env[73934]: WARNING oslo_policy.policy [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 839.883843] env[73934]: WARNING oslo_policy.policy [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 839.884786] env[73934]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 839.902066] env[73934]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Session token - acquiring [ 839.902907] env[73934]: DEBUG networking_nsxv3.common.locking [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73934) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 839.903107] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 839.903233] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 839.903600] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 839.913186] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 839.913443] env[73934]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 839.913587] env[73934]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73934) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 841.921831] env[73934]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Session token - acquiring [ 841.921831] env[73934]: DEBUG networking_nsxv3.common.locking [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73934) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 841.921831] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 841.922208] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 841.922208] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 841.930750] env[73934]: DEBUG oslo_concurrency.lockutils [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73934) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 841.931112] env[73934]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 841.931352] env[73934]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73934) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 843.932930] env[73934]: CRITICAL neutron [None req-eac9a73c-b4fb-4916-99c8-3106c24e6203 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 843.932930] env[73934]: ERROR neutron Traceback (most recent call last): [ 843.932930] env[73934]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 843.932930] env[73934]: ERROR neutron sys.exit(main()) [ 843.932930] env[73934]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 843.932930] env[73934]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 843.932930] env[73934]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 843.932930] env[73934]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 843.932930] env[73934]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 843.932930] env[73934]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 843.932930] env[73934]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 843.932930] env[73934]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 843.932930] env[73934]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 843.932930] env[73934]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 843.932930] env[73934]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 843.932930] env[73934]: ERROR neutron response = self.get(path=path, params=params) [ 843.933614] env[73934]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 843.933614] env[73934]: ERROR neutron raise RuntimeError(msg, last_err) [ 843.933614] env[73934]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 843.933614] env[73934]: ERROR neutron [ 846.492408] env[74152]: INFO neutron.common.config [-] Logging enabled! [ 846.492731] env[74152]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 846.492930] env[74152]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74152) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 846.493158] env[74152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 846.493229] env[74152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 846.494072] env[74152]: WARNING oslo_policy.policy [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 846.494291] env[74152]: WARNING oslo_policy.policy [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 846.495471] env[74152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 846.509688] env[74152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Session token - acquiring [ 846.510493] env[74152]: DEBUG networking_nsxv3.common.locking [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74152) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 846.510661] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 846.510785] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 846.511139] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 846.520242] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 846.520492] env[74152]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 846.520633] env[74152]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74152) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 848.527671] env[74152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Session token - acquiring [ 848.528268] env[74152]: DEBUG networking_nsxv3.common.locking [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74152) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 848.528476] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 848.528599] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 848.528870] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 848.538686] env[74152]: DEBUG oslo_concurrency.lockutils [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 848.538893] env[74152]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 848.539029] env[74152]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74152) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 850.540900] env[74152]: CRITICAL neutron [None req-ff592192-b7aa-436d-884a-7696858bf01f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 850.540900] env[74152]: ERROR neutron Traceback (most recent call last): [ 850.540900] env[74152]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 850.540900] env[74152]: ERROR neutron sys.exit(main()) [ 850.540900] env[74152]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 850.540900] env[74152]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 850.540900] env[74152]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 850.540900] env[74152]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 850.540900] env[74152]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 850.540900] env[74152]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 850.540900] env[74152]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 850.540900] env[74152]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 850.540900] env[74152]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 850.540900] env[74152]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 850.540900] env[74152]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 850.540900] env[74152]: ERROR neutron response = self.get(path=path, params=params) [ 850.542085] env[74152]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 850.542085] env[74152]: ERROR neutron raise RuntimeError(msg, last_err) [ 850.542085] env[74152]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 850.542085] env[74152]: ERROR neutron [ 853.300277] env[74280]: INFO neutron.common.config [-] Logging enabled! [ 853.300600] env[74280]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 853.300859] env[74280]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74280) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 853.301087] env[74280]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 853.301174] env[74280]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 853.302103] env[74280]: WARNING oslo_policy.policy [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 853.302314] env[74280]: WARNING oslo_policy.policy [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 853.303589] env[74280]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 853.318475] env[74280]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Session token - acquiring [ 853.319408] env[74280]: DEBUG networking_nsxv3.common.locking [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74280) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 853.319562] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 853.319710] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 853.320136] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 853.330386] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 853.330672] env[74280]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 853.330903] env[74280]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74280) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 855.336902] env[74280]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Session token - acquiring [ 855.337152] env[74280]: DEBUG networking_nsxv3.common.locking [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74280) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 855.337340] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 855.337460] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 855.337726] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 855.347888] env[74280]: DEBUG oslo_concurrency.lockutils [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74280) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 855.348173] env[74280]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 855.348314] env[74280]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74280) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 857.350521] env[74280]: CRITICAL neutron [None req-7ad96bea-a3b9-45d5-b658-ae8e814f6b71 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 857.350521] env[74280]: ERROR neutron Traceback (most recent call last): [ 857.350521] env[74280]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 857.350521] env[74280]: ERROR neutron sys.exit(main()) [ 857.350521] env[74280]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 857.350521] env[74280]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 857.350521] env[74280]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 857.350521] env[74280]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 857.350521] env[74280]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 857.350521] env[74280]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 857.350521] env[74280]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 857.350521] env[74280]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 857.350521] env[74280]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 857.350521] env[74280]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 857.350521] env[74280]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 857.350521] env[74280]: ERROR neutron response = self.get(path=path, params=params) [ 857.351287] env[74280]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 857.351287] env[74280]: ERROR neutron raise RuntimeError(msg, last_err) [ 857.351287] env[74280]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 857.351287] env[74280]: ERROR neutron [ 859.937056] env[74391]: INFO neutron.common.config [-] Logging enabled! [ 859.937387] env[74391]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 859.937591] env[74391]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74391) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 859.937828] env[74391]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 859.937952] env[74391]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 859.938777] env[74391]: WARNING oslo_policy.policy [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 859.938997] env[74391]: WARNING oslo_policy.policy [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 859.940261] env[74391]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 859.954568] env[74391]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Session token - acquiring [ 859.955384] env[74391]: DEBUG networking_nsxv3.common.locking [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74391) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 859.955581] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 859.955706] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 859.956075] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 859.965277] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 859.965549] env[74391]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 859.965700] env[74391]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74391) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 861.973272] env[74391]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Session token - acquiring [ 861.973981] env[74391]: DEBUG networking_nsxv3.common.locking [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74391) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 861.974542] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 861.974819] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 861.975251] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 861.985510] env[74391]: DEBUG oslo_concurrency.lockutils [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74391) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 861.985900] env[74391]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 861.986187] env[74391]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74391) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 863.988952] env[74391]: CRITICAL neutron [None req-4b9d2b3f-9326-43e6-9e2f-ae9ee7582372 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 863.988952] env[74391]: ERROR neutron Traceback (most recent call last): [ 863.988952] env[74391]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 863.988952] env[74391]: ERROR neutron sys.exit(main()) [ 863.988952] env[74391]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 863.988952] env[74391]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 863.988952] env[74391]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 863.988952] env[74391]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 863.988952] env[74391]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 863.988952] env[74391]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 863.988952] env[74391]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 863.988952] env[74391]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 863.988952] env[74391]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 863.988952] env[74391]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 863.988952] env[74391]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 863.988952] env[74391]: ERROR neutron response = self.get(path=path, params=params) [ 863.989664] env[74391]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 863.989664] env[74391]: ERROR neutron raise RuntimeError(msg, last_err) [ 863.989664] env[74391]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 863.989664] env[74391]: ERROR neutron [ 866.510006] env[74403]: INFO neutron.common.config [-] Logging enabled! [ 866.510358] env[74403]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 866.510530] env[74403]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74403) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 866.510756] env[74403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 866.510852] env[74403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 866.511694] env[74403]: WARNING oslo_policy.policy [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 866.511906] env[74403]: WARNING oslo_policy.policy [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 866.513186] env[74403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 866.527239] env[74403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Session token - acquiring [ 866.528019] env[74403]: DEBUG networking_nsxv3.common.locking [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74403) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 866.528231] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 866.528362] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 866.528739] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 866.538102] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 866.538353] env[74403]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 866.538498] env[74403]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74403) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 868.546156] env[74403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Session token - acquiring [ 868.546563] env[74403]: DEBUG networking_nsxv3.common.locking [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74403) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 868.547348] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 868.547348] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 868.547348] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 868.556821] env[74403]: DEBUG oslo_concurrency.lockutils [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 868.557068] env[74403]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 868.557209] env[74403]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74403) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 870.560154] env[74403]: CRITICAL neutron [None req-5c1e2d56-c4f1-4e78-93da-9795ecd0a896 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 870.560154] env[74403]: ERROR neutron Traceback (most recent call last): [ 870.560154] env[74403]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 870.560154] env[74403]: ERROR neutron sys.exit(main()) [ 870.560154] env[74403]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 870.560154] env[74403]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 870.560154] env[74403]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 870.560154] env[74403]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 870.560154] env[74403]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 870.560154] env[74403]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 870.560154] env[74403]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 870.560154] env[74403]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 870.560154] env[74403]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 870.560154] env[74403]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 870.560154] env[74403]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 870.560154] env[74403]: ERROR neutron response = self.get(path=path, params=params) [ 870.561007] env[74403]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 870.561007] env[74403]: ERROR neutron raise RuntimeError(msg, last_err) [ 870.561007] env[74403]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 870.561007] env[74403]: ERROR neutron [ 873.056087] env[74588]: INFO neutron.common.config [-] Logging enabled! [ 873.056425] env[74588]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 873.056614] env[74588]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74588) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 873.056850] env[74588]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 873.056975] env[74588]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 873.057879] env[74588]: WARNING oslo_policy.policy [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 873.058109] env[74588]: WARNING oslo_policy.policy [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 873.059357] env[74588]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 873.074258] env[74588]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Session token - acquiring [ 873.075063] env[74588]: DEBUG networking_nsxv3.common.locking [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74588) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 873.075254] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 873.075379] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 873.075742] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 873.084854] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 873.085109] env[74588]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 873.085180] env[74588]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74588) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 875.091166] env[74588]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Session token - acquiring [ 875.091480] env[74588]: DEBUG networking_nsxv3.common.locking [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74588) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 875.091726] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 875.091864] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 875.092171] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 875.101474] env[74588]: DEBUG oslo_concurrency.lockutils [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74588) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 875.101729] env[74588]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 875.101902] env[74588]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74588) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 877.104759] env[74588]: CRITICAL neutron [None req-f71fa571-e7b5-4aa7-ade8-abf1b40acded None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 877.104759] env[74588]: ERROR neutron Traceback (most recent call last): [ 877.104759] env[74588]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 877.104759] env[74588]: ERROR neutron sys.exit(main()) [ 877.104759] env[74588]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 877.104759] env[74588]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 877.104759] env[74588]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 877.104759] env[74588]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 877.104759] env[74588]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 877.104759] env[74588]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 877.104759] env[74588]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 877.104759] env[74588]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 877.104759] env[74588]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 877.104759] env[74588]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 877.104759] env[74588]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 877.104759] env[74588]: ERROR neutron response = self.get(path=path, params=params) [ 877.105409] env[74588]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 877.105409] env[74588]: ERROR neutron raise RuntimeError(msg, last_err) [ 877.105409] env[74588]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 877.105409] env[74588]: ERROR neutron [ 879.705746] env[74629]: INFO neutron.common.config [-] Logging enabled! [ 879.706062] env[74629]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 879.706272] env[74629]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74629) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 879.706516] env[74629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 879.706610] env[74629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 879.707469] env[74629]: WARNING oslo_policy.policy [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 879.707689] env[74629]: WARNING oslo_policy.policy [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 879.708983] env[74629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 879.723587] env[74629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Session token - acquiring [ 879.724395] env[74629]: DEBUG networking_nsxv3.common.locking [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74629) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 879.724596] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 879.724724] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 879.725149] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 879.734626] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 879.734869] env[74629]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 879.735011] env[74629]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74629) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 881.741188] env[74629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Session token - acquiring [ 881.741463] env[74629]: DEBUG networking_nsxv3.common.locking [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74629) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 881.741463] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 881.741556] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 881.741948] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 881.751084] env[74629]: DEBUG oslo_concurrency.lockutils [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 881.751288] env[74629]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 881.751424] env[74629]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74629) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 883.752919] env[74629]: CRITICAL neutron [None req-f6757ff3-5f34-4cc0-a537-f6f8656f7acb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 883.752919] env[74629]: ERROR neutron Traceback (most recent call last): [ 883.752919] env[74629]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 883.752919] env[74629]: ERROR neutron sys.exit(main()) [ 883.752919] env[74629]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 883.752919] env[74629]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 883.752919] env[74629]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 883.752919] env[74629]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 883.752919] env[74629]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 883.752919] env[74629]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 883.752919] env[74629]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 883.752919] env[74629]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 883.752919] env[74629]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 883.752919] env[74629]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 883.752919] env[74629]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 883.752919] env[74629]: ERROR neutron response = self.get(path=path, params=params) [ 883.753550] env[74629]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 883.753550] env[74629]: ERROR neutron raise RuntimeError(msg, last_err) [ 883.753550] env[74629]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 883.753550] env[74629]: ERROR neutron [ 886.460123] env[74689]: INFO neutron.common.config [-] Logging enabled! [ 886.460431] env[74689]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 886.460649] env[74689]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74689) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 886.460895] env[74689]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 886.460984] env[74689]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 886.461831] env[74689]: WARNING oslo_policy.policy [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 886.462056] env[74689]: WARNING oslo_policy.policy [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 886.463308] env[74689]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 886.478183] env[74689]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Session token - acquiring [ 886.478988] env[74689]: DEBUG networking_nsxv3.common.locking [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74689) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 886.479186] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 886.479308] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 886.479677] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 886.488746] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 886.488985] env[74689]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 886.489126] env[74689]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74689) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 888.497395] env[74689]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Session token - acquiring [ 888.497395] env[74689]: DEBUG networking_nsxv3.common.locking [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74689) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 888.497395] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 888.497736] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 888.497736] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 888.505469] env[74689]: DEBUG oslo_concurrency.lockutils [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74689) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 888.505910] env[74689]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 888.506217] env[74689]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74689) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 890.509289] env[74689]: CRITICAL neutron [None req-c0a9401e-b398-43bc-b18a-afbfa9e8dc5f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 890.509289] env[74689]: ERROR neutron Traceback (most recent call last): [ 890.509289] env[74689]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 890.509289] env[74689]: ERROR neutron sys.exit(main()) [ 890.509289] env[74689]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 890.509289] env[74689]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 890.509289] env[74689]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 890.509289] env[74689]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 890.509289] env[74689]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 890.509289] env[74689]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 890.509289] env[74689]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 890.509289] env[74689]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 890.509289] env[74689]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 890.509289] env[74689]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 890.509289] env[74689]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 890.509289] env[74689]: ERROR neutron response = self.get(path=path, params=params) [ 890.510707] env[74689]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 890.510707] env[74689]: ERROR neutron raise RuntimeError(msg, last_err) [ 890.510707] env[74689]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 890.510707] env[74689]: ERROR neutron [ 893.247129] env[74914]: INFO neutron.common.config [-] Logging enabled! [ 893.247129] env[74914]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 893.247129] env[74914]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74914) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 893.247129] env[74914]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 893.247129] env[74914]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 893.247129] env[74914]: WARNING oslo_policy.policy [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 893.247129] env[74914]: WARNING oslo_policy.policy [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 893.247129] env[74914]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 893.259526] env[74914]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Session token - acquiring [ 893.260398] env[74914]: DEBUG networking_nsxv3.common.locking [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74914) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 893.260607] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 893.260737] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 893.261098] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 893.270134] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 893.270390] env[74914]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 893.270540] env[74914]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74914) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 895.277774] env[74914]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Session token - acquiring [ 895.278156] env[74914]: DEBUG networking_nsxv3.common.locking [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74914) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 895.278355] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 895.278488] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 895.278766] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 895.288043] env[74914]: DEBUG oslo_concurrency.lockutils [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74914) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 895.288291] env[74914]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 895.288433] env[74914]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74914) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 897.291342] env[74914]: CRITICAL neutron [None req-52fe8742-6700-4c40-b45d-7695d55872dc None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 897.291342] env[74914]: ERROR neutron Traceback (most recent call last): [ 897.291342] env[74914]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 897.291342] env[74914]: ERROR neutron sys.exit(main()) [ 897.291342] env[74914]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 897.291342] env[74914]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 897.291342] env[74914]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 897.291342] env[74914]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 897.291342] env[74914]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 897.291342] env[74914]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 897.291342] env[74914]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 897.291342] env[74914]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 897.291342] env[74914]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 897.291342] env[74914]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 897.291342] env[74914]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 897.291342] env[74914]: ERROR neutron response = self.get(path=path, params=params) [ 897.292158] env[74914]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 897.292158] env[74914]: ERROR neutron raise RuntimeError(msg, last_err) [ 897.292158] env[74914]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 897.292158] env[74914]: ERROR neutron [ 899.995742] env[75036]: INFO neutron.common.config [-] Logging enabled! [ 899.995742] env[75036]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 899.995742] env[75036]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75036) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 899.995742] env[75036]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 899.995742] env[75036]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 899.995742] env[75036]: WARNING oslo_policy.policy [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 899.995742] env[75036]: WARNING oslo_policy.policy [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 899.995742] env[75036]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 900.009248] env[75036]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Session token - acquiring [ 900.010101] env[75036]: DEBUG networking_nsxv3.common.locking [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75036) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 900.010315] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 900.010444] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 900.010822] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 900.019765] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 900.020035] env[75036]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 900.020182] env[75036]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75036) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 902.026290] env[75036]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Session token - acquiring [ 902.026683] env[75036]: DEBUG networking_nsxv3.common.locking [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75036) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 902.026902] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 902.027030] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 902.027308] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 902.037906] env[75036]: DEBUG oslo_concurrency.lockutils [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75036) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 902.038170] env[75036]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 902.038315] env[75036]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75036) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 904.040922] env[75036]: CRITICAL neutron [None req-5642718d-403d-4ead-8e1b-3b72d8665b1d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 904.040922] env[75036]: ERROR neutron Traceback (most recent call last): [ 904.040922] env[75036]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 904.040922] env[75036]: ERROR neutron sys.exit(main()) [ 904.040922] env[75036]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 904.040922] env[75036]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 904.040922] env[75036]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 904.040922] env[75036]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 904.040922] env[75036]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 904.040922] env[75036]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 904.040922] env[75036]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 904.040922] env[75036]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 904.040922] env[75036]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 904.040922] env[75036]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 904.040922] env[75036]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 904.040922] env[75036]: ERROR neutron response = self.get(path=path, params=params) [ 904.041759] env[75036]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 904.041759] env[75036]: ERROR neutron raise RuntimeError(msg, last_err) [ 904.041759] env[75036]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 904.041759] env[75036]: ERROR neutron [ 906.673825] env[75143]: INFO neutron.common.config [-] Logging enabled! [ 906.674230] env[75143]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 906.674351] env[75143]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75143) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 906.674575] env[75143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 906.674669] env[75143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 906.675491] env[75143]: WARNING oslo_policy.policy [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 906.675720] env[75143]: WARNING oslo_policy.policy [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 906.676954] env[75143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 906.691487] env[75143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Session token - acquiring [ 906.692339] env[75143]: DEBUG networking_nsxv3.common.locking [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75143) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 906.692551] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 906.692687] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 906.693061] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 906.702138] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 906.702453] env[75143]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 906.702620] env[75143]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75143) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 908.708667] env[75143]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Session token - acquiring [ 908.709011] env[75143]: DEBUG networking_nsxv3.common.locking [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75143) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 908.709242] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 908.709374] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 908.709698] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 908.718956] env[75143]: DEBUG oslo_concurrency.lockutils [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75143) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 908.719202] env[75143]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 908.719346] env[75143]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75143) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 910.720974] env[75143]: CRITICAL neutron [None req-40c5ef76-d409-42bf-99e6-b85a96bf967c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 910.720974] env[75143]: ERROR neutron Traceback (most recent call last): [ 910.720974] env[75143]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 910.720974] env[75143]: ERROR neutron sys.exit(main()) [ 910.720974] env[75143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 910.720974] env[75143]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 910.720974] env[75143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 910.720974] env[75143]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 910.720974] env[75143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 910.720974] env[75143]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 910.720974] env[75143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 910.720974] env[75143]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 910.720974] env[75143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 910.720974] env[75143]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 910.720974] env[75143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 910.720974] env[75143]: ERROR neutron response = self.get(path=path, params=params) [ 910.721787] env[75143]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 910.721787] env[75143]: ERROR neutron raise RuntimeError(msg, last_err) [ 910.721787] env[75143]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 910.721787] env[75143]: ERROR neutron [ 913.395919] env[75206]: INFO neutron.common.config [-] Logging enabled! [ 913.396305] env[75206]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 913.396638] env[75206]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75206) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 913.396870] env[75206]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 913.396957] env[75206]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 913.397990] env[75206]: WARNING oslo_policy.policy [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 913.398210] env[75206]: WARNING oslo_policy.policy [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 913.399660] env[75206]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 913.416446] env[75206]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Session token - acquiring [ 913.417674] env[75206]: DEBUG networking_nsxv3.common.locking [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75206) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 913.417902] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 913.418027] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 913.418597] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 913.428257] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 913.428544] env[75206]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 913.428666] env[75206]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75206) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 915.437240] env[75206]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Session token - acquiring [ 915.437240] env[75206]: DEBUG networking_nsxv3.common.locking [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75206) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 915.437240] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 915.437797] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 915.437797] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 915.448795] env[75206]: DEBUG oslo_concurrency.lockutils [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75206) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 915.448795] env[75206]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 915.448795] env[75206]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75206) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 917.451643] env[75206]: CRITICAL neutron [None req-fcbba3c0-ba46-47a1-a5df-67333910016e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 917.451643] env[75206]: ERROR neutron Traceback (most recent call last): [ 917.451643] env[75206]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 917.451643] env[75206]: ERROR neutron sys.exit(main()) [ 917.451643] env[75206]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 917.451643] env[75206]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 917.451643] env[75206]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 917.451643] env[75206]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 917.451643] env[75206]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 917.451643] env[75206]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 917.451643] env[75206]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 917.451643] env[75206]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 917.451643] env[75206]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 917.451643] env[75206]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 917.451643] env[75206]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 917.451643] env[75206]: ERROR neutron response = self.get(path=path, params=params) [ 917.452447] env[75206]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 917.452447] env[75206]: ERROR neutron raise RuntimeError(msg, last_err) [ 917.452447] env[75206]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 917.452447] env[75206]: ERROR neutron [ 920.008271] env[75457]: INFO neutron.common.config [-] Logging enabled! [ 920.008271] env[75457]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 920.008628] env[75457]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75457) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 920.008854] env[75457]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 920.008949] env[75457]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 920.009785] env[75457]: WARNING oslo_policy.policy [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 920.010003] env[75457]: WARNING oslo_policy.policy [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 920.011208] env[75457]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 920.025431] env[75457]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Session token - acquiring [ 920.026217] env[75457]: DEBUG networking_nsxv3.common.locking [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75457) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 920.026420] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 920.026545] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 920.026923] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 920.036131] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 920.036386] env[75457]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 920.036560] env[75457]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75457) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 922.042855] env[75457]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Session token - acquiring [ 922.043162] env[75457]: DEBUG networking_nsxv3.common.locking [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75457) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 922.043259] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 922.043413] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 922.043652] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 922.053728] env[75457]: DEBUG oslo_concurrency.lockutils [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75457) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 922.054232] env[75457]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 922.054232] env[75457]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75457) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 924.057179] env[75457]: CRITICAL neutron [None req-bb4be6ce-150b-432a-b2e4-aff9592a6539 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 924.057179] env[75457]: ERROR neutron Traceback (most recent call last): [ 924.057179] env[75457]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 924.057179] env[75457]: ERROR neutron sys.exit(main()) [ 924.057179] env[75457]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 924.057179] env[75457]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 924.057179] env[75457]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 924.057179] env[75457]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 924.057179] env[75457]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 924.057179] env[75457]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 924.057179] env[75457]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 924.057179] env[75457]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 924.057179] env[75457]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 924.057179] env[75457]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 924.057179] env[75457]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 924.057179] env[75457]: ERROR neutron response = self.get(path=path, params=params) [ 924.057926] env[75457]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 924.057926] env[75457]: ERROR neutron raise RuntimeError(msg, last_err) [ 924.057926] env[75457]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 924.057926] env[75457]: ERROR neutron [ 926.755619] env[75700]: INFO neutron.common.config [-] Logging enabled! [ 926.755970] env[75700]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 926.756227] env[75700]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75700) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 926.756492] env[75700]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 926.756590] env[75700]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 926.757444] env[75700]: WARNING oslo_policy.policy [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 926.757659] env[75700]: WARNING oslo_policy.policy [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 926.758871] env[75700]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 926.775185] env[75700]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Session token - acquiring [ 926.776335] env[75700]: DEBUG networking_nsxv3.common.locking [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75700) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 926.776536] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 926.776659] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 926.777043] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 926.788268] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 926.788532] env[75700]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 926.788675] env[75700]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75700) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 928.794760] env[75700]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Session token - acquiring [ 928.795100] env[75700]: DEBUG networking_nsxv3.common.locking [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75700) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 928.795294] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 928.795426] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 928.795700] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 928.804689] env[75700]: DEBUG oslo_concurrency.lockutils [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75700) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 928.804933] env[75700]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 928.805347] env[75700]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75700) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 930.807998] env[75700]: CRITICAL neutron [None req-9a195f6b-6c16-404d-af59-7f09ded8f6f6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 930.807998] env[75700]: ERROR neutron Traceback (most recent call last): [ 930.807998] env[75700]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 930.807998] env[75700]: ERROR neutron sys.exit(main()) [ 930.807998] env[75700]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 930.807998] env[75700]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 930.807998] env[75700]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 930.807998] env[75700]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 930.807998] env[75700]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 930.807998] env[75700]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 930.807998] env[75700]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 930.807998] env[75700]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 930.807998] env[75700]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 930.807998] env[75700]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 930.807998] env[75700]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 930.807998] env[75700]: ERROR neutron response = self.get(path=path, params=params) [ 930.808927] env[75700]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 930.808927] env[75700]: ERROR neutron raise RuntimeError(msg, last_err) [ 930.808927] env[75700]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 930.808927] env[75700]: ERROR neutron [ 933.442427] env[75799]: INFO neutron.common.config [-] Logging enabled! [ 933.442748] env[75799]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 933.442988] env[75799]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75799) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 933.443240] env[75799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 933.443334] env[75799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 933.444178] env[75799]: WARNING oslo_policy.policy [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 933.444467] env[75799]: WARNING oslo_policy.policy [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 933.446028] env[75799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 933.460635] env[75799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Session token - acquiring [ 933.461425] env[75799]: DEBUG networking_nsxv3.common.locking [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75799) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 933.461622] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 933.461783] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 933.462180] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 933.471264] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 933.471517] env[75799]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 933.471663] env[75799]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75799) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 935.477835] env[75799]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Session token - acquiring [ 935.478162] env[75799]: DEBUG networking_nsxv3.common.locking [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75799) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 935.478387] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 935.478510] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 935.478786] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 935.487982] env[75799]: DEBUG oslo_concurrency.lockutils [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75799) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 935.488246] env[75799]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 935.488398] env[75799]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75799) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 937.491301] env[75799]: CRITICAL neutron [None req-1e09d003-fb91-40ae-8f1f-b09444b8c0b6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 937.491301] env[75799]: ERROR neutron Traceback (most recent call last): [ 937.491301] env[75799]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 937.491301] env[75799]: ERROR neutron sys.exit(main()) [ 937.491301] env[75799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 937.491301] env[75799]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 937.491301] env[75799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 937.491301] env[75799]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 937.491301] env[75799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 937.491301] env[75799]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 937.491301] env[75799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 937.491301] env[75799]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 937.491301] env[75799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 937.491301] env[75799]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 937.491301] env[75799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 937.491301] env[75799]: ERROR neutron response = self.get(path=path, params=params) [ 937.492148] env[75799]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 937.492148] env[75799]: ERROR neutron raise RuntimeError(msg, last_err) [ 937.492148] env[75799]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 937.492148] env[75799]: ERROR neutron [ 939.926142] env[75840]: INFO neutron.common.config [-] Logging enabled! [ 939.926506] env[75840]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 939.926755] env[75840]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75840) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 939.926983] env[75840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 939.927078] env[75840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 939.927897] env[75840]: WARNING oslo_policy.policy [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 939.928140] env[75840]: WARNING oslo_policy.policy [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 939.929326] env[75840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 939.944796] env[75840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Session token - acquiring [ 939.945569] env[75840]: DEBUG networking_nsxv3.common.locking [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75840) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 939.945776] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 939.945906] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 939.946273] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 939.955502] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 939.955745] env[75840]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 939.955890] env[75840]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75840) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 941.962146] env[75840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Session token - acquiring [ 941.962491] env[75840]: DEBUG networking_nsxv3.common.locking [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75840) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 941.962705] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 941.962852] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 941.963143] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 941.972187] env[75840]: DEBUG oslo_concurrency.lockutils [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75840) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 941.972439] env[75840]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 941.972578] env[75840]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75840) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 943.975493] env[75840]: CRITICAL neutron [None req-95c21c67-f0f2-4e94-8074-fba7c4e7be5f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 943.975493] env[75840]: ERROR neutron Traceback (most recent call last): [ 943.975493] env[75840]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 943.975493] env[75840]: ERROR neutron sys.exit(main()) [ 943.975493] env[75840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 943.975493] env[75840]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 943.975493] env[75840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 943.975493] env[75840]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 943.975493] env[75840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 943.975493] env[75840]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 943.975493] env[75840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 943.975493] env[75840]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 943.975493] env[75840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 943.975493] env[75840]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 943.975493] env[75840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 943.975493] env[75840]: ERROR neutron response = self.get(path=path, params=params) [ 943.976239] env[75840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 943.976239] env[75840]: ERROR neutron raise RuntimeError(msg, last_err) [ 943.976239] env[75840]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 943.976239] env[75840]: ERROR neutron [ 946.504685] env[75856]: INFO neutron.common.config [-] Logging enabled! [ 946.505009] env[75856]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 946.505225] env[75856]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75856) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 946.505450] env[75856]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 946.505573] env[75856]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 946.506362] env[75856]: WARNING oslo_policy.policy [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 946.506583] env[75856]: WARNING oslo_policy.policy [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 946.508283] env[75856]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 946.527402] env[75856]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Session token - acquiring [ 946.528237] env[75856]: DEBUG networking_nsxv3.common.locking [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75856) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 946.528446] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 946.528576] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 946.528941] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 946.538662] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 946.538919] env[75856]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 946.539065] env[75856]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75856) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 948.545288] env[75856]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Session token - acquiring [ 948.545747] env[75856]: DEBUG networking_nsxv3.common.locking [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75856) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 948.545989] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 948.546085] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 948.546360] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 948.555948] env[75856]: DEBUG oslo_concurrency.lockutils [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75856) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 948.556107] env[75856]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 948.556249] env[75856]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75856) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 950.559134] env[75856]: CRITICAL neutron [None req-545b7ed5-ca95-413f-b8d0-ff195373a2b5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 950.559134] env[75856]: ERROR neutron Traceback (most recent call last): [ 950.559134] env[75856]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 950.559134] env[75856]: ERROR neutron sys.exit(main()) [ 950.559134] env[75856]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 950.559134] env[75856]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 950.559134] env[75856]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 950.559134] env[75856]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 950.559134] env[75856]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 950.559134] env[75856]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 950.559134] env[75856]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 950.559134] env[75856]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 950.559134] env[75856]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 950.559134] env[75856]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 950.559134] env[75856]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 950.559134] env[75856]: ERROR neutron response = self.get(path=path, params=params) [ 950.560171] env[75856]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 950.560171] env[75856]: ERROR neutron raise RuntimeError(msg, last_err) [ 950.560171] env[75856]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 950.560171] env[75856]: ERROR neutron [ 953.284494] env[75967]: INFO neutron.common.config [-] Logging enabled! [ 953.284809] env[75967]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 953.285014] env[75967]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75967) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 953.285244] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 953.285338] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 953.286185] env[75967]: WARNING oslo_policy.policy [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 953.286411] env[75967]: WARNING oslo_policy.policy [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 953.287617] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 953.302073] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Session token - acquiring [ 953.302878] env[75967]: DEBUG networking_nsxv3.common.locking [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75967) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 953.303078] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 953.303205] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 953.303575] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 953.312788] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 953.313033] env[75967]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 953.313175] env[75967]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75967) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 955.319342] env[75967]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Session token - acquiring [ 955.319732] env[75967]: DEBUG networking_nsxv3.common.locking [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75967) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 955.319732] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 955.319845] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 955.320078] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 955.329698] env[75967]: DEBUG oslo_concurrency.lockutils [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75967) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 955.329942] env[75967]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 955.330077] env[75967]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75967) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 957.332927] env[75967]: CRITICAL neutron [None req-16e3860f-aab4-4e39-b465-b4ae2f321dc9 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 957.332927] env[75967]: ERROR neutron Traceback (most recent call last): [ 957.332927] env[75967]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 957.332927] env[75967]: ERROR neutron sys.exit(main()) [ 957.332927] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 957.332927] env[75967]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 957.332927] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 957.332927] env[75967]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 957.332927] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 957.332927] env[75967]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 957.332927] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 957.332927] env[75967]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 957.332927] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 957.332927] env[75967]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 957.332927] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 957.332927] env[75967]: ERROR neutron response = self.get(path=path, params=params) [ 957.333734] env[75967]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 957.333734] env[75967]: ERROR neutron raise RuntimeError(msg, last_err) [ 957.333734] env[75967]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 957.333734] env[75967]: ERROR neutron [ 959.935594] env[76199]: INFO neutron.common.config [-] Logging enabled! [ 959.935970] env[76199]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 959.936122] env[76199]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76199) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 959.936349] env[76199]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 959.936439] env[76199]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 959.937261] env[76199]: WARNING oslo_policy.policy [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 959.937483] env[76199]: WARNING oslo_policy.policy [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 959.938684] env[76199]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 959.952996] env[76199]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Session token - acquiring [ 959.953754] env[76199]: DEBUG networking_nsxv3.common.locking [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76199) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 959.953957] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 959.954083] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 959.954445] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 959.963547] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 959.963788] env[76199]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 959.963936] env[76199]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76199) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 961.970040] env[76199]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Session token - acquiring [ 961.970409] env[76199]: DEBUG networking_nsxv3.common.locking [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76199) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 961.970614] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 961.970791] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 961.971094] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 961.980033] env[76199]: DEBUG oslo_concurrency.lockutils [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76199) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 961.980257] env[76199]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 961.980398] env[76199]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76199) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 963.983308] env[76199]: CRITICAL neutron [None req-adf2f602-a700-4ad2-b2f7-064420b7cf0d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 963.983308] env[76199]: ERROR neutron Traceback (most recent call last): [ 963.983308] env[76199]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 963.983308] env[76199]: ERROR neutron sys.exit(main()) [ 963.983308] env[76199]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 963.983308] env[76199]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 963.983308] env[76199]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 963.983308] env[76199]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 963.983308] env[76199]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 963.983308] env[76199]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 963.983308] env[76199]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 963.983308] env[76199]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 963.983308] env[76199]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 963.983308] env[76199]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 963.983308] env[76199]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 963.983308] env[76199]: ERROR neutron response = self.get(path=path, params=params) [ 963.984144] env[76199]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 963.984144] env[76199]: ERROR neutron raise RuntimeError(msg, last_err) [ 963.984144] env[76199]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 963.984144] env[76199]: ERROR neutron [ 966.736872] env[76286]: INFO neutron.common.config [-] Logging enabled! [ 966.737216] env[76286]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 966.737376] env[76286]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76286) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 966.737611] env[76286]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 966.737707] env[76286]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 966.738558] env[76286]: WARNING oslo_policy.policy [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 966.738781] env[76286]: WARNING oslo_policy.policy [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 966.739998] env[76286]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 966.753940] env[76286]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Session token - acquiring [ 966.754734] env[76286]: DEBUG networking_nsxv3.common.locking [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76286) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 966.754935] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 966.755062] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 966.755431] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 966.764643] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 966.764884] env[76286]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 966.765032] env[76286]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76286) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 968.771074] env[76286]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Session token - acquiring [ 968.771972] env[76286]: DEBUG networking_nsxv3.common.locking [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76286) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 968.772382] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 968.772632] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 968.773028] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 968.782424] env[76286]: DEBUG oslo_concurrency.lockutils [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76286) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 968.782785] env[76286]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 968.783055] env[76286]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76286) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 970.784958] env[76286]: CRITICAL neutron [None req-eed4c482-420d-4236-bb55-6406999c7786 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 970.784958] env[76286]: ERROR neutron Traceback (most recent call last): [ 970.784958] env[76286]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 970.784958] env[76286]: ERROR neutron sys.exit(main()) [ 970.784958] env[76286]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 970.784958] env[76286]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 970.784958] env[76286]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 970.784958] env[76286]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 970.784958] env[76286]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 970.784958] env[76286]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 970.784958] env[76286]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 970.784958] env[76286]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 970.784958] env[76286]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 970.784958] env[76286]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 970.784958] env[76286]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 970.784958] env[76286]: ERROR neutron response = self.get(path=path, params=params) [ 970.785805] env[76286]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 970.785805] env[76286]: ERROR neutron raise RuntimeError(msg, last_err) [ 970.785805] env[76286]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 970.785805] env[76286]: ERROR neutron [ 973.428082] env[76332]: INFO neutron.common.config [-] Logging enabled! [ 973.428500] env[76332]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 973.428634] env[76332]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76332) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 973.428865] env[76332]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 973.428961] env[76332]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 973.429823] env[76332]: WARNING oslo_policy.policy [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 973.430044] env[76332]: WARNING oslo_policy.policy [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 973.431275] env[76332]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 973.450756] env[76332]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Session token - acquiring [ 973.451822] env[76332]: DEBUG networking_nsxv3.common.locking [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76332) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 973.452135] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 973.452303] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 973.452794] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 973.463847] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 973.464158] env[76332]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 973.464306] env[76332]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76332) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 975.470314] env[76332]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Session token - acquiring [ 975.470779] env[76332]: DEBUG networking_nsxv3.common.locking [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76332) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 975.471053] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 975.471183] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 975.471459] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 975.480640] env[76332]: DEBUG oslo_concurrency.lockutils [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76332) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 975.480943] env[76332]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 975.481147] env[76332]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76332) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 977.484417] env[76332]: CRITICAL neutron [None req-ad18d176-af13-4717-9d1d-092da6f81f8a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 977.484417] env[76332]: ERROR neutron Traceback (most recent call last): [ 977.484417] env[76332]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 977.484417] env[76332]: ERROR neutron sys.exit(main()) [ 977.484417] env[76332]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 977.484417] env[76332]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 977.484417] env[76332]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 977.484417] env[76332]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 977.484417] env[76332]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 977.484417] env[76332]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 977.484417] env[76332]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 977.484417] env[76332]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 977.484417] env[76332]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 977.484417] env[76332]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 977.484417] env[76332]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 977.484417] env[76332]: ERROR neutron response = self.get(path=path, params=params) [ 977.485222] env[76332]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 977.485222] env[76332]: ERROR neutron raise RuntimeError(msg, last_err) [ 977.485222] env[76332]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 977.485222] env[76332]: ERROR neutron [ 979.863442] env[76461]: INFO neutron.common.config [-] Logging enabled! [ 979.863750] env[76461]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 979.863954] env[76461]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76461) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 979.865282] env[76461]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 979.865282] env[76461]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 979.865282] env[76461]: WARNING oslo_policy.policy [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 979.865282] env[76461]: WARNING oslo_policy.policy [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 979.866474] env[76461]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 979.880498] env[76461]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Session token - acquiring [ 979.881245] env[76461]: DEBUG networking_nsxv3.common.locking [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76461) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 979.881446] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 979.881571] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 979.881968] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 979.890786] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 979.891035] env[76461]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 979.891169] env[76461]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76461) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 981.897279] env[76461]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Session token - acquiring [ 981.897764] env[76461]: DEBUG networking_nsxv3.common.locking [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76461) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 981.897965] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 981.898082] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 981.898354] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 981.907521] env[76461]: DEBUG oslo_concurrency.lockutils [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76461) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 981.907754] env[76461]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 981.907894] env[76461]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76461) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 983.909269] env[76461]: CRITICAL neutron [None req-0c94be3d-1cb4-430b-b182-f8644e9ddbcc None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 983.909269] env[76461]: ERROR neutron Traceback (most recent call last): [ 983.909269] env[76461]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 983.909269] env[76461]: ERROR neutron sys.exit(main()) [ 983.909269] env[76461]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 983.909269] env[76461]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 983.909269] env[76461]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 983.909269] env[76461]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 983.909269] env[76461]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 983.909269] env[76461]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 983.909269] env[76461]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 983.909269] env[76461]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 983.909269] env[76461]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 983.909269] env[76461]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 983.909269] env[76461]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 983.909269] env[76461]: ERROR neutron response = self.get(path=path, params=params) [ 983.910252] env[76461]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 983.910252] env[76461]: ERROR neutron raise RuntimeError(msg, last_err) [ 983.910252] env[76461]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 983.910252] env[76461]: ERROR neutron [ 986.400929] env[76471]: INFO neutron.common.config [-] Logging enabled! [ 986.401307] env[76471]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 986.401454] env[76471]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76471) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 986.401701] env[76471]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 986.401820] env[76471]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 986.402624] env[76471]: WARNING oslo_policy.policy [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 986.402846] env[76471]: WARNING oslo_policy.policy [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 986.403987] env[76471]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 986.418355] env[76471]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Session token - acquiring [ 986.419106] env[76471]: DEBUG networking_nsxv3.common.locking [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76471) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 986.419305] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 986.419435] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 986.419801] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 986.428576] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 986.428820] env[76471]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 986.428971] env[76471]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76471) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 988.435267] env[76471]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Session token - acquiring [ 988.435785] env[76471]: DEBUG networking_nsxv3.common.locking [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76471) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 988.435916] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 988.435979] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 988.436280] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 988.445348] env[76471]: DEBUG oslo_concurrency.lockutils [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76471) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 988.445589] env[76471]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 988.445838] env[76471]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76471) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 990.448731] env[76471]: CRITICAL neutron [None req-26b8eb66-b2cb-411e-a45b-cbb3fddbde0a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 990.448731] env[76471]: ERROR neutron Traceback (most recent call last): [ 990.448731] env[76471]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 990.448731] env[76471]: ERROR neutron sys.exit(main()) [ 990.448731] env[76471]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 990.448731] env[76471]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 990.448731] env[76471]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 990.448731] env[76471]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 990.448731] env[76471]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 990.448731] env[76471]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 990.448731] env[76471]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 990.448731] env[76471]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 990.448731] env[76471]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 990.448731] env[76471]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 990.448731] env[76471]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 990.448731] env[76471]: ERROR neutron response = self.get(path=path, params=params) [ 990.449613] env[76471]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 990.449613] env[76471]: ERROR neutron raise RuntimeError(msg, last_err) [ 990.449613] env[76471]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 990.449613] env[76471]: ERROR neutron [ 992.961867] env[76500]: INFO neutron.common.config [-] Logging enabled! [ 992.962191] env[76500]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 992.962412] env[76500]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76500) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 992.962633] env[76500]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 992.962724] env[76500]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 992.963565] env[76500]: WARNING oslo_policy.policy [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 992.963778] env[76500]: WARNING oslo_policy.policy [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 992.965038] env[76500]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 992.980869] env[76500]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Session token - acquiring [ 992.982205] env[76500]: DEBUG networking_nsxv3.common.locking [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76500) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 992.983192] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 992.983368] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 992.984336] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 992.997718] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 992.998022] env[76500]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 992.998166] env[76500]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76500) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 995.004398] env[76500]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Session token - acquiring [ 995.004742] env[76500]: DEBUG networking_nsxv3.common.locking [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76500) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 995.004742] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 995.004886] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 995.005171] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 995.014443] env[76500]: DEBUG oslo_concurrency.lockutils [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76500) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 995.014678] env[76500]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 995.014816] env[76500]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76500) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 997.016890] env[76500]: CRITICAL neutron [None req-fe654926-a76f-4cc0-be39-55705ad49560 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 997.016890] env[76500]: ERROR neutron Traceback (most recent call last): [ 997.016890] env[76500]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 997.016890] env[76500]: ERROR neutron sys.exit(main()) [ 997.016890] env[76500]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 997.016890] env[76500]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 997.016890] env[76500]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 997.016890] env[76500]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 997.016890] env[76500]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 997.016890] env[76500]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 997.016890] env[76500]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 997.016890] env[76500]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 997.016890] env[76500]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 997.016890] env[76500]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 997.016890] env[76500]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 997.016890] env[76500]: ERROR neutron response = self.get(path=path, params=params) [ 997.018039] env[76500]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 997.018039] env[76500]: ERROR neutron raise RuntimeError(msg, last_err) [ 997.018039] env[76500]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 997.018039] env[76500]: ERROR neutron [ 999.413405] env[76528]: INFO neutron.common.config [-] Logging enabled! [ 999.413801] env[76528]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 999.413916] env[76528]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76528) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 999.414175] env[76528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 999.414281] env[76528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 999.415135] env[76528]: WARNING oslo_policy.policy [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 999.419106] env[76528]: WARNING oslo_policy.policy [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 999.419106] env[76528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 999.436335] env[76528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Session token - acquiring [ 999.437202] env[76528]: DEBUG networking_nsxv3.common.locking [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76528) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 999.437392] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 999.437555] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 999.437902] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 999.448792] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 999.449091] env[76528]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 999.449256] env[76528]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76528) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1001.455301] env[76528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Session token - acquiring [ 1001.455645] env[76528]: DEBUG networking_nsxv3.common.locking [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76528) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1001.455799] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1001.455925] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1001.456234] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1001.465745] env[76528]: DEBUG oslo_concurrency.lockutils [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1001.466038] env[76528]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1001.466219] env[76528]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76528) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1003.471042] env[76528]: CRITICAL neutron [None req-1c8125cd-c38c-4da1-8c4a-930a8d709e31 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1003.471042] env[76528]: ERROR neutron Traceback (most recent call last): [ 1003.471042] env[76528]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1003.471042] env[76528]: ERROR neutron sys.exit(main()) [ 1003.471042] env[76528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1003.471042] env[76528]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1003.471042] env[76528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1003.471042] env[76528]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1003.471042] env[76528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1003.471042] env[76528]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1003.471042] env[76528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1003.471042] env[76528]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1003.471042] env[76528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1003.471042] env[76528]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1003.471042] env[76528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1003.471042] env[76528]: ERROR neutron response = self.get(path=path, params=params) [ 1003.471838] env[76528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1003.471838] env[76528]: ERROR neutron raise RuntimeError(msg, last_err) [ 1003.471838] env[76528]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1003.471838] env[76528]: ERROR neutron [ 1005.871480] env[76590]: INFO neutron.common.config [-] Logging enabled! [ 1005.871879] env[76590]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 1005.872038] env[76590]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76590) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1005.872260] env[76590]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1005.872352] env[76590]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1005.873187] env[76590]: WARNING oslo_policy.policy [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1005.873401] env[76590]: WARNING oslo_policy.policy [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1005.874556] env[76590]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1005.888597] env[76590]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Session token - acquiring [ 1005.889353] env[76590]: DEBUG networking_nsxv3.common.locking [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76590) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1005.889550] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1005.889677] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1005.890028] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1005.898794] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1005.899036] env[76590]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1005.899229] env[76590]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76590) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1007.905331] env[76590]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Session token - acquiring [ 1007.905678] env[76590]: DEBUG networking_nsxv3.common.locking [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76590) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1007.905910] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1007.906052] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1007.906331] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1007.915263] env[76590]: DEBUG oslo_concurrency.lockutils [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76590) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1007.915490] env[76590]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1007.915631] env[76590]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76590) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1009.916891] env[76590]: CRITICAL neutron [None req-9066f97f-8358-4894-9165-3fefe6116cfc None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1009.916891] env[76590]: ERROR neutron Traceback (most recent call last): [ 1009.916891] env[76590]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1009.916891] env[76590]: ERROR neutron sys.exit(main()) [ 1009.916891] env[76590]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1009.916891] env[76590]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1009.916891] env[76590]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1009.916891] env[76590]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1009.916891] env[76590]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1009.916891] env[76590]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1009.916891] env[76590]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1009.916891] env[76590]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1009.916891] env[76590]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1009.916891] env[76590]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1009.916891] env[76590]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1009.916891] env[76590]: ERROR neutron response = self.get(path=path, params=params) [ 1009.917630] env[76590]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1009.917630] env[76590]: ERROR neutron raise RuntimeError(msg, last_err) [ 1009.917630] env[76590]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1009.917630] env[76590]: ERROR neutron [ 1012.331298] env[76606]: INFO neutron.common.config [-] Logging enabled! [ 1012.331695] env[76606]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 1012.331788] env[76606]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76606) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1012.332013] env[76606]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1012.332147] env[76606]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1012.332967] env[76606]: WARNING oslo_policy.policy [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1012.333190] env[76606]: WARNING oslo_policy.policy [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1012.334392] env[76606]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1012.349005] env[76606]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Session token - acquiring [ 1012.349992] env[76606]: DEBUG networking_nsxv3.common.locking [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76606) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1012.350233] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1012.350385] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1012.350757] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1012.359913] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1012.360192] env[76606]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1012.360341] env[76606]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76606) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1014.366358] env[76606]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Session token - acquiring [ 1014.366768] env[76606]: DEBUG networking_nsxv3.common.locking [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76606) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1014.367037] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1014.367194] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1014.367469] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1014.376392] env[76606]: DEBUG oslo_concurrency.lockutils [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76606) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1014.376631] env[76606]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1014.376770] env[76606]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76606) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1016.379751] env[76606]: CRITICAL neutron [None req-89a226e2-3f1a-4963-bacf-bb1670511827 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1016.379751] env[76606]: ERROR neutron Traceback (most recent call last): [ 1016.379751] env[76606]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1016.379751] env[76606]: ERROR neutron sys.exit(main()) [ 1016.379751] env[76606]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1016.379751] env[76606]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1016.379751] env[76606]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1016.379751] env[76606]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1016.379751] env[76606]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1016.379751] env[76606]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1016.379751] env[76606]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1016.379751] env[76606]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1016.379751] env[76606]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1016.379751] env[76606]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1016.379751] env[76606]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1016.379751] env[76606]: ERROR neutron response = self.get(path=path, params=params) [ 1016.380817] env[76606]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1016.380817] env[76606]: ERROR neutron raise RuntimeError(msg, last_err) [ 1016.380817] env[76606]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1016.380817] env[76606]: ERROR neutron [ 1018.832334] env[76654]: INFO neutron.common.config [-] Logging enabled! [ 1018.832840] env[76654]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 1018.832840] env[76654]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76654) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1018.833064] env[76654]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1018.833158] env[76654]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1018.833980] env[76654]: WARNING oslo_policy.policy [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1018.834243] env[76654]: WARNING oslo_policy.policy [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1018.835409] env[76654]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1018.850275] env[76654]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Session token - acquiring [ 1018.851042] env[76654]: DEBUG networking_nsxv3.common.locking [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76654) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1018.851240] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1018.851364] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1018.851769] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1018.860684] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1018.860946] env[76654]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1018.861109] env[76654]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76654) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1020.867047] env[76654]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Session token - acquiring [ 1020.867539] env[76654]: DEBUG networking_nsxv3.common.locking [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76654) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1020.867635] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1020.867736] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1020.868016] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1020.876883] env[76654]: DEBUG oslo_concurrency.lockutils [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76654) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1020.877112] env[76654]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1020.877248] env[76654]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76654) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1022.880251] env[76654]: CRITICAL neutron [None req-c6247086-e051-4e48-b2f4-b2775f1d4359 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1022.880251] env[76654]: ERROR neutron Traceback (most recent call last): [ 1022.880251] env[76654]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1022.880251] env[76654]: ERROR neutron sys.exit(main()) [ 1022.880251] env[76654]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1022.880251] env[76654]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1022.880251] env[76654]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1022.880251] env[76654]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1022.880251] env[76654]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1022.880251] env[76654]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1022.880251] env[76654]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1022.880251] env[76654]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1022.880251] env[76654]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1022.880251] env[76654]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1022.880251] env[76654]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1022.880251] env[76654]: ERROR neutron response = self.get(path=path, params=params) [ 1022.881532] env[76654]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1022.881532] env[76654]: ERROR neutron raise RuntimeError(msg, last_err) [ 1022.881532] env[76654]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1022.881532] env[76654]: ERROR neutron [ 1025.346153] env[76661]: INFO neutron.common.config [-] Logging enabled! [ 1025.346600] env[76661]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 1025.346704] env[76661]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76661) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1025.346876] env[76661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1025.347013] env[76661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1025.347789] env[76661]: WARNING oslo_policy.policy [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1025.348026] env[76661]: WARNING oslo_policy.policy [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1025.349207] env[76661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1025.363286] env[76661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Session token - acquiring [ 1025.364109] env[76661]: DEBUG networking_nsxv3.common.locking [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76661) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1025.364336] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1025.364465] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1025.364820] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1025.374244] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1025.374489] env[76661]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1025.374636] env[76661]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76661) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1027.389583] env[76661]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Session token - acquiring [ 1027.390313] env[76661]: DEBUG networking_nsxv3.common.locking [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76661) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1027.390680] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1027.390919] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1027.391287] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1027.400449] env[76661]: DEBUG oslo_concurrency.lockutils [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76661) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1027.400844] env[76661]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1027.401112] env[76661]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76661) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1029.404354] env[76661]: CRITICAL neutron [None req-09b28f3a-b148-49fd-80eb-2dc83eaa06da None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1029.404354] env[76661]: ERROR neutron Traceback (most recent call last): [ 1029.404354] env[76661]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1029.404354] env[76661]: ERROR neutron sys.exit(main()) [ 1029.404354] env[76661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1029.404354] env[76661]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1029.404354] env[76661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1029.404354] env[76661]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1029.404354] env[76661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1029.404354] env[76661]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1029.404354] env[76661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1029.404354] env[76661]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1029.404354] env[76661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1029.404354] env[76661]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1029.404354] env[76661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1029.404354] env[76661]: ERROR neutron response = self.get(path=path, params=params) [ 1029.405170] env[76661]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1029.405170] env[76661]: ERROR neutron raise RuntimeError(msg, last_err) [ 1029.405170] env[76661]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1029.405170] env[76661]: ERROR neutron [ 1031.734373] env[76682]: INFO neutron.common.config [-] Logging enabled! [ 1031.734792] env[76682]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev62 [ 1031.734888] env[76682]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76682) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1031.735086] env[76682]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1031.735181] env[76682]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1031.736016] env[76682]: WARNING oslo_policy.policy [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1031.736260] env[76682]: WARNING oslo_policy.policy [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1031.737422] env[76682]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1031.751423] env[76682]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Session token - acquiring [ 1031.752324] env[76682]: DEBUG networking_nsxv3.common.locking [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76682) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1031.752531] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1031.752661] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1031.753108] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1031.761964] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1031.762217] env[76682]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1031.762365] env[76682]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76682) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1033.768536] env[76682]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Session token - acquiring [ 1033.769114] env[76682]: DEBUG networking_nsxv3.common.locking [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76682) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1033.769192] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1033.769305] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1033.769676] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1033.778864] env[76682]: DEBUG oslo_concurrency.lockutils [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76682) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1033.779125] env[76682]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1033.779259] env[76682]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0d246b1-800b-4ec2-bd45-6215bbb3c649 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76682) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}