[ 475.090331] env[62733]: INFO neutron.common.config [-] Logging enabled! [ 475.090805] env[62733]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 475.090805] env[62733]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62733) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 475.091023] env[62733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 475.091117] env[62733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 475.091943] env[62733]: WARNING oslo_policy.policy [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 475.092235] env[62733]: WARNING oslo_policy.policy [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 475.093408] env[62733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 475.107734] env[62733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Session token - acquiring [ 475.108575] env[62733]: DEBUG networking_nsxv3.common.locking [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62733) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 475.108787] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 475.108915] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 475.109382] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 475.119141] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 475.119419] env[62733]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 475.119563] env[62733]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62733) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 477.126034] env[62733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Session token - acquiring [ 477.126553] env[62733]: DEBUG networking_nsxv3.common.locking [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62733) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 477.126635] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 477.126758] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 477.127008] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 477.136126] env[62733]: DEBUG oslo_concurrency.lockutils [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62733) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 477.136366] env[62733]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 477.136507] env[62733]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62733) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 479.139495] env[62733]: CRITICAL neutron [None req-885dad47-de2c-4cc5-a908-69c522a26d96 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 479.139495] env[62733]: ERROR neutron Traceback (most recent call last): [ 479.139495] env[62733]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 479.139495] env[62733]: ERROR neutron sys.exit(main()) [ 479.139495] env[62733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 479.139495] env[62733]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 479.139495] env[62733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 479.139495] env[62733]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 479.139495] env[62733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 479.139495] env[62733]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 479.139495] env[62733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 479.139495] env[62733]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 479.139495] env[62733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 479.139495] env[62733]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 479.139495] env[62733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 479.139495] env[62733]: ERROR neutron response = self.get(path=path, params=params) [ 479.140584] env[62733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 479.140584] env[62733]: ERROR neutron raise RuntimeError(msg, last_err) [ 479.140584] env[62733]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 479.140584] env[62733]: ERROR neutron [ 481.518450] env[62763]: INFO neutron.common.config [-] Logging enabled! [ 481.518892] env[62763]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 481.518968] env[62763]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62763) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 481.519125] env[62763]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 481.519213] env[62763]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 481.520111] env[62763]: WARNING oslo_policy.policy [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 481.520374] env[62763]: WARNING oslo_policy.policy [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 481.521560] env[62763]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 481.535677] env[62763]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Session token - acquiring [ 481.536452] env[62763]: DEBUG networking_nsxv3.common.locking [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62763) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 481.536649] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 481.536780] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 481.537125] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 481.545956] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 481.546191] env[62763]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 481.546365] env[62763]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62763) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 483.552302] env[62763]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Session token - acquiring [ 483.552897] env[62763]: DEBUG networking_nsxv3.common.locking [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62763) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 483.552897] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 483.552990] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 483.553194] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 483.562082] env[62763]: DEBUG oslo_concurrency.lockutils [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62763) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 483.562306] env[62763]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 483.562445] env[62763]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62763) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 485.564928] env[62763]: CRITICAL neutron [None req-79fe7873-b37a-4ccd-a032-525f02be7354 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 485.564928] env[62763]: ERROR neutron Traceback (most recent call last): [ 485.564928] env[62763]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 485.564928] env[62763]: ERROR neutron sys.exit(main()) [ 485.564928] env[62763]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 485.564928] env[62763]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 485.564928] env[62763]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 485.564928] env[62763]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 485.564928] env[62763]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 485.564928] env[62763]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 485.564928] env[62763]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 485.564928] env[62763]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 485.564928] env[62763]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 485.564928] env[62763]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 485.564928] env[62763]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 485.564928] env[62763]: ERROR neutron response = self.get(path=path, params=params) [ 485.565935] env[62763]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 485.565935] env[62763]: ERROR neutron raise RuntimeError(msg, last_err) [ 485.565935] env[62763]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 485.565935] env[62763]: ERROR neutron [ 488.043145] env[62788]: INFO neutron.common.config [-] Logging enabled! [ 488.043603] env[62788]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 488.043681] env[62788]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62788) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 488.043862] env[62788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 488.043958] env[62788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 488.044828] env[62788]: WARNING oslo_policy.policy [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 488.045064] env[62788]: WARNING oslo_policy.policy [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 488.046230] env[62788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 488.060277] env[62788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Session token - acquiring [ 488.061073] env[62788]: DEBUG networking_nsxv3.common.locking [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62788) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 488.061276] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 488.061406] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 488.061764] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 488.070752] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 488.071030] env[62788]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 488.071195] env[62788]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62788) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 490.077226] env[62788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Session token - acquiring [ 490.077578] env[62788]: DEBUG networking_nsxv3.common.locking [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62788) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 490.077776] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 490.077902] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 490.078174] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 490.086862] env[62788]: DEBUG oslo_concurrency.lockutils [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62788) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 490.087091] env[62788]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 490.087236] env[62788]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62788) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 492.090197] env[62788]: CRITICAL neutron [None req-5468e63b-b4fc-4dfa-abde-c4fe08f8825c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 492.090197] env[62788]: ERROR neutron Traceback (most recent call last): [ 492.090197] env[62788]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 492.090197] env[62788]: ERROR neutron sys.exit(main()) [ 492.090197] env[62788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 492.090197] env[62788]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 492.090197] env[62788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 492.090197] env[62788]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 492.090197] env[62788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 492.090197] env[62788]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 492.090197] env[62788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 492.090197] env[62788]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 492.090197] env[62788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 492.090197] env[62788]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 492.090197] env[62788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 492.090197] env[62788]: ERROR neutron response = self.get(path=path, params=params) [ 492.090981] env[62788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 492.090981] env[62788]: ERROR neutron raise RuntimeError(msg, last_err) [ 492.090981] env[62788]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 492.090981] env[62788]: ERROR neutron [ 494.498136] env[62877]: INFO neutron.common.config [-] Logging enabled! [ 494.498581] env[62877]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 494.498621] env[62877]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62877) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 494.498830] env[62877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 494.498923] env[62877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 494.499750] env[62877]: WARNING oslo_policy.policy [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 494.499961] env[62877]: WARNING oslo_policy.policy [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 494.501147] env[62877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 494.514863] env[62877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Session token - acquiring [ 494.515600] env[62877]: DEBUG networking_nsxv3.common.locking [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62877) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 494.515799] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 494.515924] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 494.516305] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 494.525273] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 494.525512] env[62877]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 494.525660] env[62877]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62877) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 496.531689] env[62877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Session token - acquiring [ 496.532031] env[62877]: DEBUG networking_nsxv3.common.locking [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62877) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 496.532231] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 496.532362] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 496.532638] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 496.541470] env[62877]: DEBUG oslo_concurrency.lockutils [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 496.541703] env[62877]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 496.541840] env[62877]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62877) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 498.544800] env[62877]: CRITICAL neutron [None req-0c456b00-ed88-47a3-98ca-9ced5044a23e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 498.544800] env[62877]: ERROR neutron Traceback (most recent call last): [ 498.544800] env[62877]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 498.544800] env[62877]: ERROR neutron sys.exit(main()) [ 498.544800] env[62877]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 498.544800] env[62877]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 498.544800] env[62877]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 498.544800] env[62877]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 498.544800] env[62877]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 498.544800] env[62877]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 498.544800] env[62877]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 498.544800] env[62877]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 498.544800] env[62877]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 498.544800] env[62877]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 498.544800] env[62877]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 498.544800] env[62877]: ERROR neutron response = self.get(path=path, params=params) [ 498.545759] env[62877]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 498.545759] env[62877]: ERROR neutron raise RuntimeError(msg, last_err) [ 498.545759] env[62877]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 498.545759] env[62877]: ERROR neutron [ 500.990075] env[62884]: INFO neutron.common.config [-] Logging enabled! [ 500.990482] env[62884]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 500.990588] env[62884]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62884) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 500.990809] env[62884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 500.990904] env[62884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 500.991765] env[62884]: WARNING oslo_policy.policy [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 500.991993] env[62884]: WARNING oslo_policy.policy [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 500.993263] env[62884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 501.007231] env[62884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Session token - acquiring [ 501.007972] env[62884]: DEBUG networking_nsxv3.common.locking [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62884) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 501.008190] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 501.008318] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 501.008670] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 501.017177] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 501.017432] env[62884]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 501.017590] env[62884]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62884) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 503.023807] env[62884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Session token - acquiring [ 503.024157] env[62884]: DEBUG networking_nsxv3.common.locking [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62884) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 503.024321] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 503.024442] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 503.024734] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 503.033907] env[62884]: DEBUG oslo_concurrency.lockutils [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 503.034149] env[62884]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 503.034289] env[62884]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62884) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 505.037029] env[62884]: CRITICAL neutron [None req-1e7af928-1544-4b6d-af2a-98276c56951d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 505.037029] env[62884]: ERROR neutron Traceback (most recent call last): [ 505.037029] env[62884]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 505.037029] env[62884]: ERROR neutron sys.exit(main()) [ 505.037029] env[62884]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 505.037029] env[62884]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 505.037029] env[62884]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 505.037029] env[62884]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 505.037029] env[62884]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 505.037029] env[62884]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 505.037029] env[62884]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 505.037029] env[62884]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 505.037029] env[62884]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 505.037029] env[62884]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 505.037029] env[62884]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 505.037029] env[62884]: ERROR neutron response = self.get(path=path, params=params) [ 505.037875] env[62884]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 505.037875] env[62884]: ERROR neutron raise RuntimeError(msg, last_err) [ 505.037875] env[62884]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 505.037875] env[62884]: ERROR neutron [ 507.494268] env[62892]: INFO neutron.common.config [-] Logging enabled! [ 507.494613] env[62892]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 507.494996] env[62892]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62892) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 507.495232] env[62892]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 507.495328] env[62892]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 507.496199] env[62892]: WARNING oslo_policy.policy [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 507.496425] env[62892]: WARNING oslo_policy.policy [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 507.497626] env[62892]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 507.511856] env[62892]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Session token - acquiring [ 507.512612] env[62892]: DEBUG networking_nsxv3.common.locking [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62892) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 507.512809] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 507.512933] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 507.513288] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 507.522980] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 507.523217] env[62892]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 507.523366] env[62892]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62892) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 509.529441] env[62892]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Session token - acquiring [ 509.529775] env[62892]: DEBUG networking_nsxv3.common.locking [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62892) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 509.529915] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 509.530037] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 509.530302] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 509.539268] env[62892]: DEBUG oslo_concurrency.lockutils [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62892) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 509.539520] env[62892]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 509.539663] env[62892]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62892) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 511.540918] env[62892]: CRITICAL neutron [None req-e33e01b3-fd86-4c19-b416-e1f0e76d3958 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 511.540918] env[62892]: ERROR neutron Traceback (most recent call last): [ 511.540918] env[62892]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 511.540918] env[62892]: ERROR neutron sys.exit(main()) [ 511.540918] env[62892]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 511.540918] env[62892]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 511.540918] env[62892]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 511.540918] env[62892]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 511.540918] env[62892]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 511.540918] env[62892]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 511.540918] env[62892]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 511.540918] env[62892]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 511.540918] env[62892]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 511.540918] env[62892]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 511.540918] env[62892]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 511.540918] env[62892]: ERROR neutron response = self.get(path=path, params=params) [ 511.541668] env[62892]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 511.541668] env[62892]: ERROR neutron raise RuntimeError(msg, last_err) [ 511.541668] env[62892]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 511.541668] env[62892]: ERROR neutron [ 514.151571] env[63116]: INFO neutron.common.config [-] Logging enabled! [ 514.151903] env[63116]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 514.152139] env[63116]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63116) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 514.152370] env[63116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 514.152459] env[63116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 514.153332] env[63116]: WARNING oslo_policy.policy [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 514.153581] env[63116]: WARNING oslo_policy.policy [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 514.154838] env[63116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 514.171874] env[63116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Session token - acquiring [ 514.172759] env[63116]: DEBUG networking_nsxv3.common.locking [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63116) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 514.172961] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 514.173083] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 514.173446] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 514.183663] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 514.183928] env[63116]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 514.184090] env[63116]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63116) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 516.191293] env[63116]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Session token - acquiring [ 516.191719] env[63116]: DEBUG networking_nsxv3.common.locking [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63116) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 516.191975] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 516.192172] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 516.192577] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 516.204026] env[63116]: DEBUG oslo_concurrency.lockutils [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63116) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 516.204387] env[63116]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 516.204598] env[63116]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63116) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 518.207762] env[63116]: CRITICAL neutron [None req-89ae3e5f-0980-45be-8320-df42f7ce1ddf None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 518.207762] env[63116]: ERROR neutron Traceback (most recent call last): [ 518.207762] env[63116]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 518.207762] env[63116]: ERROR neutron sys.exit(main()) [ 518.207762] env[63116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 518.207762] env[63116]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 518.207762] env[63116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 518.207762] env[63116]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 518.207762] env[63116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 518.207762] env[63116]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 518.207762] env[63116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 518.207762] env[63116]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 518.207762] env[63116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 518.207762] env[63116]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 518.207762] env[63116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 518.207762] env[63116]: ERROR neutron response = self.get(path=path, params=params) [ 518.208920] env[63116]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 518.208920] env[63116]: ERROR neutron raise RuntimeError(msg, last_err) [ 518.208920] env[63116]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 518.208920] env[63116]: ERROR neutron [ 520.986419] env[63341]: INFO neutron.common.config [-] Logging enabled! [ 520.986651] env[63341]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 520.986985] env[63341]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63341) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 520.987275] env[63341]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 520.987357] env[63341]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 520.989046] env[63341]: WARNING oslo_policy.policy [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 520.989367] env[63341]: WARNING oslo_policy.policy [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 520.991247] env[63341]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 521.007626] env[63341]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Session token - acquiring [ 521.008024] env[63341]: DEBUG networking_nsxv3.common.locking [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63341) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 521.008271] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 521.008420] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 521.008806] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 521.021097] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 521.021097] env[63341]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 521.021097] env[63341]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63341) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 523.025466] env[63341]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Session token - acquiring [ 523.025808] env[63341]: DEBUG networking_nsxv3.common.locking [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63341) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 523.026052] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 523.026140] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 523.026413] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 523.036821] env[63341]: DEBUG oslo_concurrency.lockutils [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63341) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 523.036907] env[63341]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 523.037025] env[63341]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63341) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 525.039573] env[63341]: CRITICAL neutron [None req-aa25e8a6-cd19-4824-9585-2e8be10f041b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 525.039573] env[63341]: ERROR neutron Traceback (most recent call last): [ 525.039573] env[63341]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 525.039573] env[63341]: ERROR neutron sys.exit(main()) [ 525.039573] env[63341]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 525.039573] env[63341]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 525.039573] env[63341]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 525.039573] env[63341]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 525.039573] env[63341]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 525.039573] env[63341]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 525.039573] env[63341]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 525.039573] env[63341]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 525.039573] env[63341]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 525.039573] env[63341]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 525.039573] env[63341]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 525.039573] env[63341]: ERROR neutron response = self.get(path=path, params=params) [ 525.040390] env[63341]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 525.040390] env[63341]: ERROR neutron raise RuntimeError(msg, last_err) [ 525.040390] env[63341]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 525.040390] env[63341]: ERROR neutron [ 527.793384] env[63988]: INFO neutron.common.config [-] Logging enabled! [ 527.793691] env[63988]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 527.794021] env[63988]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63988) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 527.794265] env[63988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 527.794361] env[63988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 527.795299] env[63988]: WARNING oslo_policy.policy [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 527.795521] env[63988]: WARNING oslo_policy.policy [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 527.796849] env[63988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 527.812942] env[63988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Session token - acquiring [ 527.813827] env[63988]: DEBUG networking_nsxv3.common.locking [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63988) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 527.814024] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 527.814146] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 527.814510] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 527.824162] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 527.824517] env[63988]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 527.824667] env[63988]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63988) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 529.831342] env[63988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Session token - acquiring [ 529.831666] env[63988]: DEBUG networking_nsxv3.common.locking [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63988) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 529.831855] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 529.831975] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 529.832311] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 529.842307] env[63988]: DEBUG oslo_concurrency.lockutils [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 529.842307] env[63988]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 529.842435] env[63988]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63988) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 531.845190] env[63988]: CRITICAL neutron [None req-f84e4146-a90d-4ce4-8693-8b91a808af02 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 531.845190] env[63988]: ERROR neutron Traceback (most recent call last): [ 531.845190] env[63988]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 531.845190] env[63988]: ERROR neutron sys.exit(main()) [ 531.845190] env[63988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 531.845190] env[63988]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 531.845190] env[63988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 531.845190] env[63988]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 531.845190] env[63988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 531.845190] env[63988]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 531.845190] env[63988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 531.845190] env[63988]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 531.845190] env[63988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 531.845190] env[63988]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 531.845190] env[63988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 531.845190] env[63988]: ERROR neutron response = self.get(path=path, params=params) [ 531.846258] env[63988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 531.846258] env[63988]: ERROR neutron raise RuntimeError(msg, last_err) [ 531.846258] env[63988]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 531.846258] env[63988]: ERROR neutron [ 534.938977] env[64559]: INFO neutron.common.config [-] Logging enabled! [ 534.938977] env[64559]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 534.939403] env[64559]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64559) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 534.939403] env[64559]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 534.939494] env[64559]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 534.940442] env[64559]: WARNING oslo_policy.policy [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 534.940654] env[64559]: WARNING oslo_policy.policy [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 534.941943] env[64559]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 534.964661] env[64559]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Session token - acquiring [ 534.965561] env[64559]: DEBUG networking_nsxv3.common.locking [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64559) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 534.965763] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 534.965891] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 534.966564] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 534.979177] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 534.979284] env[64559]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 534.979472] env[64559]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64559) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 536.985459] env[64559]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Session token - acquiring [ 536.985865] env[64559]: DEBUG networking_nsxv3.common.locking [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64559) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 536.986015] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 536.986087] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 536.986384] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 536.996354] env[64559]: DEBUG oslo_concurrency.lockutils [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64559) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 536.996613] env[64559]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 536.996946] env[64559]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64559) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 538.999592] env[64559]: CRITICAL neutron [None req-770a7295-6c20-417e-8b44-b7d94f613ce8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 538.999592] env[64559]: ERROR neutron Traceback (most recent call last): [ 538.999592] env[64559]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 538.999592] env[64559]: ERROR neutron sys.exit(main()) [ 538.999592] env[64559]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 538.999592] env[64559]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 538.999592] env[64559]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 538.999592] env[64559]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 538.999592] env[64559]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 538.999592] env[64559]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 538.999592] env[64559]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 538.999592] env[64559]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 538.999592] env[64559]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 538.999592] env[64559]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 538.999592] env[64559]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 538.999592] env[64559]: ERROR neutron response = self.get(path=path, params=params) [ 539.000318] env[64559]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 539.000318] env[64559]: ERROR neutron raise RuntimeError(msg, last_err) [ 539.000318] env[64559]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 539.000318] env[64559]: ERROR neutron [ 541.783062] env[64988]: INFO neutron.common.config [-] Logging enabled! [ 541.783417] env[64988]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 541.783683] env[64988]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64988) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 541.783944] env[64988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 541.784043] env[64988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 541.785713] env[64988]: WARNING oslo_policy.policy [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 541.785713] env[64988]: WARNING oslo_policy.policy [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 541.786640] env[64988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 541.802288] env[64988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Session token - acquiring [ 541.803211] env[64988]: DEBUG networking_nsxv3.common.locking [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64988) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 541.803425] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 541.803566] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 541.803923] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 541.816854] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 541.816854] env[64988]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 541.816854] env[64988]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64988) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 543.823200] env[64988]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Session token - acquiring [ 543.823200] env[64988]: DEBUG networking_nsxv3.common.locking [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64988) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 543.823200] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 543.823534] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 543.823534] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 543.833492] env[64988]: DEBUG oslo_concurrency.lockutils [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64988) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 543.834000] env[64988]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 543.834392] env[64988]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64988) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 545.846812] env[64988]: CRITICAL neutron [None req-005c74ce-677d-44c8-bd7f-f6d7edc25b57 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 545.846812] env[64988]: ERROR neutron Traceback (most recent call last): [ 545.846812] env[64988]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 545.846812] env[64988]: ERROR neutron sys.exit(main()) [ 545.846812] env[64988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 545.846812] env[64988]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 545.846812] env[64988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 545.846812] env[64988]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 545.846812] env[64988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 545.846812] env[64988]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 545.846812] env[64988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 545.846812] env[64988]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 545.846812] env[64988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 545.846812] env[64988]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 545.846812] env[64988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 545.846812] env[64988]: ERROR neutron response = self.get(path=path, params=params) [ 545.849066] env[64988]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 545.849066] env[64988]: ERROR neutron raise RuntimeError(msg, last_err) [ 545.849066] env[64988]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 545.849066] env[64988]: ERROR neutron [ 548.766470] env[65429]: INFO neutron.common.config [-] Logging enabled! [ 548.766778] env[65429]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 548.767016] env[65429]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65429) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 548.767265] env[65429]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 548.767820] env[65429]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 548.768225] env[65429]: WARNING oslo_policy.policy [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 548.768448] env[65429]: WARNING oslo_policy.policy [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 548.769734] env[65429]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 548.784071] env[65429]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Session token - acquiring [ 548.787995] env[65429]: DEBUG networking_nsxv3.common.locking [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65429) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 548.787995] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 548.788518] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 548.788518] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 548.797121] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 548.797426] env[65429]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 548.797589] env[65429]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65429) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 550.805889] env[65429]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Session token - acquiring [ 550.805889] env[65429]: DEBUG networking_nsxv3.common.locking [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65429) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 550.805889] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 550.806464] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 550.806464] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 550.814351] env[65429]: DEBUG oslo_concurrency.lockutils [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65429) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 550.814830] env[65429]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 550.815107] env[65429]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65429) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 552.816904] env[65429]: CRITICAL neutron [None req-193ba511-3694-46b0-b0c4-226612048a54 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 552.816904] env[65429]: ERROR neutron Traceback (most recent call last): [ 552.816904] env[65429]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 552.816904] env[65429]: ERROR neutron sys.exit(main()) [ 552.816904] env[65429]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 552.816904] env[65429]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 552.816904] env[65429]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 552.816904] env[65429]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 552.816904] env[65429]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 552.816904] env[65429]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 552.816904] env[65429]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 552.816904] env[65429]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 552.816904] env[65429]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 552.816904] env[65429]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 552.816904] env[65429]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 552.816904] env[65429]: ERROR neutron response = self.get(path=path, params=params) [ 552.817565] env[65429]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 552.817565] env[65429]: ERROR neutron raise RuntimeError(msg, last_err) [ 552.817565] env[65429]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 552.817565] env[65429]: ERROR neutron [ 555.580283] env[65778]: INFO neutron.common.config [-] Logging enabled! [ 555.580565] env[65778]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 555.580844] env[65778]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65778) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 555.581074] env[65778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 555.581163] env[65778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 555.582029] env[65778]: WARNING oslo_policy.policy [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 555.582252] env[65778]: WARNING oslo_policy.policy [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 555.583553] env[65778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 555.597702] env[65778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Session token - acquiring [ 555.598566] env[65778]: DEBUG networking_nsxv3.common.locking [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65778) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 555.598771] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 555.598893] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 555.599314] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 555.608731] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 555.609632] env[65778]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 555.609632] env[65778]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65778) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 557.615233] env[65778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Session token - acquiring [ 557.615551] env[65778]: DEBUG networking_nsxv3.common.locking [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65778) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 557.615743] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 557.615862] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 557.616150] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 557.626507] env[65778]: DEBUG oslo_concurrency.lockutils [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65778) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 557.626507] env[65778]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 557.626636] env[65778]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65778) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 559.628969] env[65778]: CRITICAL neutron [None req-b3a7ff47-4067-4cd6-ad37-ea560db04b4a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 559.628969] env[65778]: ERROR neutron Traceback (most recent call last): [ 559.628969] env[65778]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 559.628969] env[65778]: ERROR neutron sys.exit(main()) [ 559.628969] env[65778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 559.628969] env[65778]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 559.628969] env[65778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 559.628969] env[65778]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 559.628969] env[65778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 559.628969] env[65778]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 559.628969] env[65778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 559.628969] env[65778]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 559.628969] env[65778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 559.628969] env[65778]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 559.628969] env[65778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 559.628969] env[65778]: ERROR neutron response = self.get(path=path, params=params) [ 559.629837] env[65778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 559.629837] env[65778]: ERROR neutron raise RuntimeError(msg, last_err) [ 559.629837] env[65778]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 559.629837] env[65778]: ERROR neutron [ 562.594942] env[65979]: INFO neutron.common.config [-] Logging enabled! [ 562.595209] env[65979]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 562.595565] env[65979]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65979) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 562.595800] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 562.595889] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 562.596892] env[65979]: WARNING oslo_policy.policy [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 562.597159] env[65979]: WARNING oslo_policy.policy [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 562.599713] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 562.614392] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Session token - acquiring [ 562.615413] env[65979]: DEBUG networking_nsxv3.common.locking [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65979) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 562.615707] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 562.615835] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 562.616308] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 562.635037] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 562.635516] env[65979]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 562.635724] env[65979]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65979) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 564.644302] env[65979]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Session token - acquiring [ 564.644302] env[65979]: DEBUG networking_nsxv3.common.locking [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65979) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 564.644302] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 564.644691] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 564.644691] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 564.652775] env[65979]: DEBUG oslo_concurrency.lockutils [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65979) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 564.653167] env[65979]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 564.654283] env[65979]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65979) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 566.656384] env[65979]: CRITICAL neutron [None req-225c9853-7bcb-4211-ad7b-e7bed23b190e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 566.656384] env[65979]: ERROR neutron Traceback (most recent call last): [ 566.656384] env[65979]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 566.656384] env[65979]: ERROR neutron sys.exit(main()) [ 566.656384] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 566.656384] env[65979]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 566.656384] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 566.656384] env[65979]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 566.656384] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 566.656384] env[65979]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 566.656384] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 566.656384] env[65979]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 566.656384] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 566.656384] env[65979]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 566.656384] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 566.656384] env[65979]: ERROR neutron response = self.get(path=path, params=params) [ 566.657096] env[65979]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 566.657096] env[65979]: ERROR neutron raise RuntimeError(msg, last_err) [ 566.657096] env[65979]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 566.657096] env[65979]: ERROR neutron [ 569.524427] env[66123]: INFO neutron.common.config [-] Logging enabled! [ 569.524669] env[66123]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 569.524974] env[66123]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66123) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 569.525196] env[66123]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 569.525294] env[66123]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 569.526161] env[66123]: WARNING oslo_policy.policy [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 569.526374] env[66123]: WARNING oslo_policy.policy [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 569.527612] env[66123]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 569.542367] env[66123]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Session token - acquiring [ 569.543152] env[66123]: DEBUG networking_nsxv3.common.locking [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66123) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 569.543417] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 569.543637] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 569.544083] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 569.553822] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 569.554136] env[66123]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 569.554362] env[66123]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66123) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 571.560817] env[66123]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Session token - acquiring [ 571.561134] env[66123]: DEBUG networking_nsxv3.common.locking [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66123) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 571.561328] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 571.561445] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 571.561715] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 571.577682] env[66123]: DEBUG oslo_concurrency.lockutils [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66123) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 571.577793] env[66123]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 571.577911] env[66123]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66123) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 573.580893] env[66123]: CRITICAL neutron [None req-41dc22de-4141-426d-8e07-3299ce6e56a0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 573.580893] env[66123]: ERROR neutron Traceback (most recent call last): [ 573.580893] env[66123]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 573.580893] env[66123]: ERROR neutron sys.exit(main()) [ 573.580893] env[66123]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 573.580893] env[66123]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 573.580893] env[66123]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 573.580893] env[66123]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 573.580893] env[66123]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 573.580893] env[66123]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 573.580893] env[66123]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 573.580893] env[66123]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 573.580893] env[66123]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 573.580893] env[66123]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 573.580893] env[66123]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 573.580893] env[66123]: ERROR neutron response = self.get(path=path, params=params) [ 573.581804] env[66123]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 573.581804] env[66123]: ERROR neutron raise RuntimeError(msg, last_err) [ 573.581804] env[66123]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 573.581804] env[66123]: ERROR neutron [ 576.433333] env[66784]: INFO neutron.common.config [-] Logging enabled! [ 576.433576] env[66784]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 576.433789] env[66784]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66784) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 576.434015] env[66784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 576.434109] env[66784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 576.434961] env[66784]: WARNING oslo_policy.policy [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 576.435191] env[66784]: WARNING oslo_policy.policy [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 576.436429] env[66784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 576.450967] env[66784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Session token - acquiring [ 576.451803] env[66784]: DEBUG networking_nsxv3.common.locking [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66784) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 576.452029] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 576.452161] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 576.452533] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 576.461763] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 576.462015] env[66784]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 576.462159] env[66784]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66784) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 578.468690] env[66784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Session token - acquiring [ 578.468690] env[66784]: DEBUG networking_nsxv3.common.locking [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66784) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 578.469016] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 578.469016] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 578.469205] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 578.479067] env[66784]: DEBUG oslo_concurrency.lockutils [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 578.479338] env[66784]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 578.479977] env[66784]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66784) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 580.481755] env[66784]: CRITICAL neutron [None req-ed294b9a-0477-488f-bfd8-864f35be0cbd None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 580.481755] env[66784]: ERROR neutron Traceback (most recent call last): [ 580.481755] env[66784]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 580.481755] env[66784]: ERROR neutron sys.exit(main()) [ 580.481755] env[66784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 580.481755] env[66784]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 580.481755] env[66784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 580.481755] env[66784]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 580.481755] env[66784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 580.481755] env[66784]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 580.481755] env[66784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 580.481755] env[66784]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 580.481755] env[66784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 580.481755] env[66784]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 580.481755] env[66784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 580.481755] env[66784]: ERROR neutron response = self.get(path=path, params=params) [ 580.482941] env[66784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 580.482941] env[66784]: ERROR neutron raise RuntimeError(msg, last_err) [ 580.482941] env[66784]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 580.482941] env[66784]: ERROR neutron [ 583.622265] env[67368]: INFO neutron.common.config [-] Logging enabled! [ 583.622539] env[67368]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 583.623040] env[67368]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67368) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 583.623309] env[67368]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 583.623375] env[67368]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 583.624944] env[67368]: WARNING oslo_policy.policy [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 583.624944] env[67368]: WARNING oslo_policy.policy [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 583.625959] env[67368]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 583.641640] env[67368]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Session token - acquiring [ 583.642356] env[67368]: DEBUG networking_nsxv3.common.locking [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67368) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 583.642683] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 583.642843] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 583.643274] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 583.654946] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 583.655712] env[67368]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 583.655712] env[67368]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67368) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 585.661627] env[67368]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Session token - acquiring [ 585.661967] env[67368]: DEBUG networking_nsxv3.common.locking [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67368) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 585.662171] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 585.662347] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 585.662565] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 585.671775] env[67368]: DEBUG oslo_concurrency.lockutils [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67368) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 585.672306] env[67368]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 585.672491] env[67368]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67368) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 587.675389] env[67368]: CRITICAL neutron [None req-78e12111-f8f6-46ab-9eb6-a867a2061630 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 587.675389] env[67368]: ERROR neutron Traceback (most recent call last): [ 587.675389] env[67368]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 587.675389] env[67368]: ERROR neutron sys.exit(main()) [ 587.675389] env[67368]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 587.675389] env[67368]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 587.675389] env[67368]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 587.675389] env[67368]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 587.675389] env[67368]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 587.675389] env[67368]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 587.675389] env[67368]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 587.675389] env[67368]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 587.675389] env[67368]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 587.675389] env[67368]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 587.675389] env[67368]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 587.675389] env[67368]: ERROR neutron response = self.get(path=path, params=params) [ 587.676210] env[67368]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 587.676210] env[67368]: ERROR neutron raise RuntimeError(msg, last_err) [ 587.676210] env[67368]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 587.676210] env[67368]: ERROR neutron [ 590.431613] env[67841]: INFO neutron.common.config [-] Logging enabled! [ 590.431865] env[67841]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 590.432221] env[67841]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67841) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 590.432399] env[67841]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 590.432492] env[67841]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 590.433358] env[67841]: WARNING oslo_policy.policy [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 590.433580] env[67841]: WARNING oslo_policy.policy [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 590.434903] env[67841]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 590.448952] env[67841]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Session token - acquiring [ 590.449744] env[67841]: DEBUG networking_nsxv3.common.locking [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67841) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 590.450197] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 590.450197] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 590.450421] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 590.459459] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 590.459719] env[67841]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 590.459869] env[67841]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67841) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 592.469774] env[67841]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Session token - acquiring [ 592.469774] env[67841]: DEBUG networking_nsxv3.common.locking [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67841) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 592.469774] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 592.470128] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 592.470128] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 592.476359] env[67841]: DEBUG oslo_concurrency.lockutils [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67841) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 592.476667] env[67841]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 592.476876] env[67841]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67841) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 594.479794] env[67841]: CRITICAL neutron [None req-70023dbc-b1ba-48da-a9a4-671a819fcc24 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 594.479794] env[67841]: ERROR neutron Traceback (most recent call last): [ 594.479794] env[67841]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 594.479794] env[67841]: ERROR neutron sys.exit(main()) [ 594.479794] env[67841]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 594.479794] env[67841]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 594.479794] env[67841]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 594.479794] env[67841]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 594.479794] env[67841]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 594.479794] env[67841]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 594.479794] env[67841]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 594.479794] env[67841]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 594.479794] env[67841]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 594.479794] env[67841]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 594.479794] env[67841]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 594.479794] env[67841]: ERROR neutron response = self.get(path=path, params=params) [ 594.480536] env[67841]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 594.480536] env[67841]: ERROR neutron raise RuntimeError(msg, last_err) [ 594.480536] env[67841]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 594.480536] env[67841]: ERROR neutron [ 597.240269] env[68171]: INFO neutron.common.config [-] Logging enabled! [ 597.240598] env[68171]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 597.240858] env[68171]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68171) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 597.241085] env[68171]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 597.241174] env[68171]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 597.242092] env[68171]: WARNING oslo_policy.policy [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 597.242296] env[68171]: WARNING oslo_policy.policy [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 597.243562] env[68171]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 597.258715] env[68171]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Session token - acquiring [ 597.259894] env[68171]: DEBUG networking_nsxv3.common.locking [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68171) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 597.259894] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 597.260014] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 597.260014] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 597.268924] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 597.269181] env[68171]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 597.269366] env[68171]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68171) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 599.275492] env[68171]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Session token - acquiring [ 599.275832] env[68171]: DEBUG networking_nsxv3.common.locking [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68171) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 599.276340] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 599.276340] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 599.276448] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 599.285763] env[68171]: DEBUG oslo_concurrency.lockutils [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68171) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 599.286335] env[68171]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 599.286490] env[68171]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68171) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 601.295325] env[68171]: CRITICAL neutron [None req-579ab306-24b3-41f3-afef-555faa88fec3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 601.295325] env[68171]: ERROR neutron Traceback (most recent call last): [ 601.295325] env[68171]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 601.295325] env[68171]: ERROR neutron sys.exit(main()) [ 601.295325] env[68171]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 601.295325] env[68171]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 601.295325] env[68171]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 601.295325] env[68171]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 601.295325] env[68171]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 601.295325] env[68171]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 601.295325] env[68171]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 601.295325] env[68171]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 601.295325] env[68171]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 601.295325] env[68171]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 601.295325] env[68171]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 601.295325] env[68171]: ERROR neutron response = self.get(path=path, params=params) [ 601.296221] env[68171]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 601.296221] env[68171]: ERROR neutron raise RuntimeError(msg, last_err) [ 601.296221] env[68171]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 601.296221] env[68171]: ERROR neutron [ 604.060336] env[68490]: INFO neutron.common.config [-] Logging enabled! [ 604.060666] env[68490]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 604.060903] env[68490]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68490) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 604.061136] env[68490]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 604.061286] env[68490]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 604.062079] env[68490]: WARNING oslo_policy.policy [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 604.062308] env[68490]: WARNING oslo_policy.policy [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 604.063541] env[68490]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 604.078141] env[68490]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Session token - acquiring [ 604.079011] env[68490]: DEBUG networking_nsxv3.common.locking [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68490) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 604.079241] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 604.079396] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 604.079775] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 604.089206] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 604.089496] env[68490]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 604.089652] env[68490]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68490) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 606.098202] env[68490]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Session token - acquiring [ 606.098202] env[68490]: DEBUG networking_nsxv3.common.locking [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68490) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 606.098202] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 606.098607] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 606.098607] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 606.106758] env[68490]: DEBUG oslo_concurrency.lockutils [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68490) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 606.106758] env[68490]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 606.106758] env[68490]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68490) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 608.109755] env[68490]: CRITICAL neutron [None req-914a1a7f-ed67-4226-bec6-20e2c103f6a6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 608.109755] env[68490]: ERROR neutron Traceback (most recent call last): [ 608.109755] env[68490]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 608.109755] env[68490]: ERROR neutron sys.exit(main()) [ 608.109755] env[68490]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 608.109755] env[68490]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 608.109755] env[68490]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 608.109755] env[68490]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 608.109755] env[68490]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 608.109755] env[68490]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 608.109755] env[68490]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 608.109755] env[68490]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 608.109755] env[68490]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 608.109755] env[68490]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 608.109755] env[68490]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 608.109755] env[68490]: ERROR neutron response = self.get(path=path, params=params) [ 608.110447] env[68490]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 608.110447] env[68490]: ERROR neutron raise RuntimeError(msg, last_err) [ 608.110447] env[68490]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 608.110447] env[68490]: ERROR neutron [ 610.722058] env[68732]: INFO neutron.common.config [-] Logging enabled! [ 610.722385] env[68732]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 610.722853] env[68732]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68732) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 610.722853] env[68732]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 610.722958] env[68732]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 610.723873] env[68732]: WARNING oslo_policy.policy [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 610.724177] env[68732]: WARNING oslo_policy.policy [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 610.725398] env[68732]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 610.740326] env[68732]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Session token - acquiring [ 610.741457] env[68732]: DEBUG networking_nsxv3.common.locking [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68732) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 610.741751] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 610.741960] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 610.742370] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 610.751501] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 610.751727] env[68732]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 610.751948] env[68732]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68732) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 612.758589] env[68732]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Session token - acquiring [ 612.759084] env[68732]: DEBUG networking_nsxv3.common.locking [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68732) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 612.759367] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 612.759514] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 612.759796] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 612.769068] env[68732]: DEBUG oslo_concurrency.lockutils [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68732) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 612.769439] env[68732]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 612.769494] env[68732]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68732) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 614.773484] env[68732]: CRITICAL neutron [None req-e72db325-3bc8-4c90-aa88-bab244ee28f6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 614.773484] env[68732]: ERROR neutron Traceback (most recent call last): [ 614.773484] env[68732]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 614.773484] env[68732]: ERROR neutron sys.exit(main()) [ 614.773484] env[68732]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 614.773484] env[68732]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 614.773484] env[68732]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 614.773484] env[68732]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 614.773484] env[68732]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 614.773484] env[68732]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 614.773484] env[68732]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 614.773484] env[68732]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 614.773484] env[68732]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 614.773484] env[68732]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 614.773484] env[68732]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 614.773484] env[68732]: ERROR neutron response = self.get(path=path, params=params) [ 614.774408] env[68732]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 614.774408] env[68732]: ERROR neutron raise RuntimeError(msg, last_err) [ 614.774408] env[68732]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 614.774408] env[68732]: ERROR neutron [ 617.503195] env[68909]: INFO neutron.common.config [-] Logging enabled! [ 617.503195] env[68909]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 617.503611] env[68909]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68909) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 617.503643] env[68909]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 617.504423] env[68909]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 617.504582] env[68909]: WARNING oslo_policy.policy [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 617.508642] env[68909]: WARNING oslo_policy.policy [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 617.508642] env[68909]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 617.521349] env[68909]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Session token - acquiring [ 617.522182] env[68909]: DEBUG networking_nsxv3.common.locking [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68909) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 617.522383] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 617.522508] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 617.522869] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 617.532159] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 617.532467] env[68909]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 617.532563] env[68909]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68909) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 619.541968] env[68909]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Session token - acquiring [ 619.541968] env[68909]: DEBUG networking_nsxv3.common.locking [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68909) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 619.541968] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 619.542437] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 619.542437] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 619.552559] env[68909]: DEBUG oslo_concurrency.lockutils [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68909) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 619.553022] env[68909]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 619.553488] env[68909]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68909) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 621.556264] env[68909]: CRITICAL neutron [None req-4d23476f-e3cf-4fdf-a422-e2230318d038 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 621.556264] env[68909]: ERROR neutron Traceback (most recent call last): [ 621.556264] env[68909]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 621.556264] env[68909]: ERROR neutron sys.exit(main()) [ 621.556264] env[68909]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 621.556264] env[68909]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 621.556264] env[68909]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 621.556264] env[68909]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 621.556264] env[68909]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 621.556264] env[68909]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 621.556264] env[68909]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 621.556264] env[68909]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 621.556264] env[68909]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 621.556264] env[68909]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 621.556264] env[68909]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 621.556264] env[68909]: ERROR neutron response = self.get(path=path, params=params) [ 621.556900] env[68909]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 621.556900] env[68909]: ERROR neutron raise RuntimeError(msg, last_err) [ 621.556900] env[68909]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 621.556900] env[68909]: ERROR neutron [ 624.491161] env[69362]: INFO neutron.common.config [-] Logging enabled! [ 624.491441] env[69362]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 624.491682] env[69362]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69362) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 624.491910] env[69362]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 624.492035] env[69362]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 624.492858] env[69362]: WARNING oslo_policy.policy [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 624.493078] env[69362]: WARNING oslo_policy.policy [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 624.494482] env[69362]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 624.510023] env[69362]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Session token - acquiring [ 624.510863] env[69362]: DEBUG networking_nsxv3.common.locking [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69362) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 624.511142] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 624.511286] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 624.511655] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 624.523318] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 624.523318] env[69362]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 624.523318] env[69362]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69362) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 626.527369] env[69362]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Session token - acquiring [ 626.527660] env[69362]: DEBUG networking_nsxv3.common.locking [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69362) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 626.527864] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 626.528062] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 626.528308] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 626.537781] env[69362]: DEBUG oslo_concurrency.lockutils [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69362) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 626.538078] env[69362]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 626.538250] env[69362]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69362) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 628.540912] env[69362]: CRITICAL neutron [None req-c8a11176-cfbf-4162-920d-a573d856efd0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 628.540912] env[69362]: ERROR neutron Traceback (most recent call last): [ 628.540912] env[69362]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 628.540912] env[69362]: ERROR neutron sys.exit(main()) [ 628.540912] env[69362]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 628.540912] env[69362]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 628.540912] env[69362]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 628.540912] env[69362]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 628.540912] env[69362]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 628.540912] env[69362]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 628.540912] env[69362]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 628.540912] env[69362]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 628.540912] env[69362]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 628.540912] env[69362]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 628.540912] env[69362]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 628.540912] env[69362]: ERROR neutron response = self.get(path=path, params=params) [ 628.541509] env[69362]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 628.541509] env[69362]: ERROR neutron raise RuntimeError(msg, last_err) [ 628.541509] env[69362]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 628.541509] env[69362]: ERROR neutron [ 631.296881] env[69675]: INFO neutron.common.config [-] Logging enabled! [ 631.296881] env[69675]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 631.297127] env[69675]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69675) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 631.297218] env[69675]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 631.297309] env[69675]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 631.298152] env[69675]: WARNING oslo_policy.policy [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 631.298374] env[69675]: WARNING oslo_policy.policy [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 631.299624] env[69675]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 631.314563] env[69675]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Session token - acquiring [ 631.315361] env[69675]: DEBUG networking_nsxv3.common.locking [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69675) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 631.315558] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 631.315689] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 631.316068] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 631.325977] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 631.326191] env[69675]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 631.326334] env[69675]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69675) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 633.332349] env[69675]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Session token - acquiring [ 633.332766] env[69675]: DEBUG networking_nsxv3.common.locking [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69675) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 633.332862] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 633.332958] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 633.333244] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 633.344852] env[69675]: DEBUG oslo_concurrency.lockutils [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69675) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 633.345123] env[69675]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 633.345265] env[69675]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69675) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 635.350995] env[69675]: CRITICAL neutron [None req-5f725fcd-a909-40dd-b537-b8b9b3afb7f5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 635.350995] env[69675]: ERROR neutron Traceback (most recent call last): [ 635.350995] env[69675]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 635.350995] env[69675]: ERROR neutron sys.exit(main()) [ 635.350995] env[69675]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 635.350995] env[69675]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 635.350995] env[69675]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 635.350995] env[69675]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 635.350995] env[69675]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 635.350995] env[69675]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 635.350995] env[69675]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 635.350995] env[69675]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 635.350995] env[69675]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 635.350995] env[69675]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 635.350995] env[69675]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 635.350995] env[69675]: ERROR neutron response = self.get(path=path, params=params) [ 635.351775] env[69675]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 635.351775] env[69675]: ERROR neutron raise RuntimeError(msg, last_err) [ 635.351775] env[69675]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 635.351775] env[69675]: ERROR neutron [ 638.323432] env[69874]: INFO neutron.common.config [-] Logging enabled! [ 638.323432] env[69874]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 638.323869] env[69874]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69874) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 638.324109] env[69874]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 638.324198] env[69874]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 638.325067] env[69874]: WARNING oslo_policy.policy [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 638.325280] env[69874]: WARNING oslo_policy.policy [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 638.326617] env[69874]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 638.341801] env[69874]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Session token - acquiring [ 638.342660] env[69874]: DEBUG networking_nsxv3.common.locking [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69874) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 638.342876] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 638.343002] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 638.343372] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 638.354023] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 638.354023] env[69874]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 638.354465] env[69874]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69874) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 640.361498] env[69874]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Session token - acquiring [ 640.361865] env[69874]: DEBUG networking_nsxv3.common.locking [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69874) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 640.362129] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 640.362193] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 640.362453] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 640.374680] env[69874]: DEBUG oslo_concurrency.lockutils [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69874) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 640.375006] env[69874]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 640.375148] env[69874]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69874) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 642.376902] env[69874]: CRITICAL neutron [None req-ad9cd7c6-8646-407e-9e94-2350565c07ee None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 642.376902] env[69874]: ERROR neutron Traceback (most recent call last): [ 642.376902] env[69874]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 642.376902] env[69874]: ERROR neutron sys.exit(main()) [ 642.376902] env[69874]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 642.376902] env[69874]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 642.376902] env[69874]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 642.376902] env[69874]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 642.376902] env[69874]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 642.376902] env[69874]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 642.376902] env[69874]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 642.376902] env[69874]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 642.376902] env[69874]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 642.376902] env[69874]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 642.376902] env[69874]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 642.376902] env[69874]: ERROR neutron response = self.get(path=path, params=params) [ 642.377693] env[69874]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 642.377693] env[69874]: ERROR neutron raise RuntimeError(msg, last_err) [ 642.377693] env[69874]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 642.377693] env[69874]: ERROR neutron [ 645.360078] env[70300]: INFO neutron.common.config [-] Logging enabled! [ 645.360078] env[70300]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 645.360078] env[70300]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70300) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 645.360078] env[70300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 645.360078] env[70300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 645.360078] env[70300]: WARNING oslo_policy.policy [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 645.360078] env[70300]: WARNING oslo_policy.policy [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 645.361337] env[70300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 645.375545] env[70300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Session token - acquiring [ 645.376607] env[70300]: DEBUG networking_nsxv3.common.locking [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70300) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 645.376823] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 645.376947] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 645.377329] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 645.387409] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 645.387889] env[70300]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 645.387889] env[70300]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70300) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 647.393920] env[70300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Session token - acquiring [ 647.394308] env[70300]: DEBUG networking_nsxv3.common.locking [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70300) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 647.395487] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 647.395487] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 647.395487] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 647.404210] env[70300]: DEBUG oslo_concurrency.lockutils [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 647.404448] env[70300]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 647.404587] env[70300]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70300) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 649.407634] env[70300]: CRITICAL neutron [None req-39dd360a-7788-4435-8219-622ecab3a522 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 649.407634] env[70300]: ERROR neutron Traceback (most recent call last): [ 649.407634] env[70300]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 649.407634] env[70300]: ERROR neutron sys.exit(main()) [ 649.407634] env[70300]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 649.407634] env[70300]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 649.407634] env[70300]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 649.407634] env[70300]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 649.407634] env[70300]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 649.407634] env[70300]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 649.407634] env[70300]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 649.407634] env[70300]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 649.407634] env[70300]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 649.407634] env[70300]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 649.407634] env[70300]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 649.407634] env[70300]: ERROR neutron response = self.get(path=path, params=params) [ 649.409082] env[70300]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 649.409082] env[70300]: ERROR neutron raise RuntimeError(msg, last_err) [ 649.409082] env[70300]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 649.409082] env[70300]: ERROR neutron [ 652.003030] env[70528]: INFO neutron.common.config [-] Logging enabled! [ 652.003346] env[70528]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 652.003767] env[70528]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70528) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 652.003767] env[70528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 652.003858] env[70528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 652.004777] env[70528]: WARNING oslo_policy.policy [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 652.004997] env[70528]: WARNING oslo_policy.policy [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 652.006208] env[70528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 652.020326] env[70528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Session token - acquiring [ 652.021134] env[70528]: DEBUG networking_nsxv3.common.locking [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70528) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 652.021335] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 652.021464] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 652.021855] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 652.031115] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 652.031390] env[70528]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 652.031541] env[70528]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70528) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 654.037717] env[70528]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Session token - acquiring [ 654.038021] env[70528]: DEBUG networking_nsxv3.common.locking [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70528) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 654.038209] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 654.038330] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 654.038598] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 654.048793] env[70528]: DEBUG oslo_concurrency.lockutils [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70528) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 654.049022] env[70528]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 654.049186] env[70528]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70528) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 656.052093] env[70528]: CRITICAL neutron [None req-edb9d994-0bd5-4de4-b212-84b1ec27ae83 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 656.052093] env[70528]: ERROR neutron Traceback (most recent call last): [ 656.052093] env[70528]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 656.052093] env[70528]: ERROR neutron sys.exit(main()) [ 656.052093] env[70528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 656.052093] env[70528]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 656.052093] env[70528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 656.052093] env[70528]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 656.052093] env[70528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 656.052093] env[70528]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 656.052093] env[70528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 656.052093] env[70528]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 656.052093] env[70528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 656.052093] env[70528]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 656.052093] env[70528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 656.052093] env[70528]: ERROR neutron response = self.get(path=path, params=params) [ 656.052732] env[70528]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 656.052732] env[70528]: ERROR neutron raise RuntimeError(msg, last_err) [ 656.052732] env[70528]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 656.052732] env[70528]: ERROR neutron [ 658.776862] env[70557]: INFO neutron.common.config [-] Logging enabled! [ 658.777190] env[70557]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 658.784052] env[70557]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70557) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 658.784052] env[70557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 658.784052] env[70557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 658.784052] env[70557]: WARNING oslo_policy.policy [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 658.784052] env[70557]: WARNING oslo_policy.policy [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 658.784052] env[70557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 658.795926] env[70557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Session token - acquiring [ 658.797086] env[70557]: DEBUG networking_nsxv3.common.locking [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70557) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 658.797616] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 658.797812] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 658.798354] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 658.810505] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 658.810695] env[70557]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 658.811108] env[70557]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70557) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 660.817706] env[70557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Session token - acquiring [ 660.817706] env[70557]: DEBUG networking_nsxv3.common.locking [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70557) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 660.818027] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 660.818027] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 660.818446] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 660.828357] env[70557]: DEBUG oslo_concurrency.lockutils [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 660.828357] env[70557]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 660.828357] env[70557]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70557) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 662.831287] env[70557]: CRITICAL neutron [None req-0c157fce-3ef7-4e8e-8c53-e7451b2100d5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 662.831287] env[70557]: ERROR neutron Traceback (most recent call last): [ 662.831287] env[70557]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 662.831287] env[70557]: ERROR neutron sys.exit(main()) [ 662.831287] env[70557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 662.831287] env[70557]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 662.831287] env[70557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 662.831287] env[70557]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 662.831287] env[70557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 662.831287] env[70557]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 662.831287] env[70557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 662.831287] env[70557]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 662.831287] env[70557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 662.831287] env[70557]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 662.831287] env[70557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 662.831287] env[70557]: ERROR neutron response = self.get(path=path, params=params) [ 662.832064] env[70557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 662.832064] env[70557]: ERROR neutron raise RuntimeError(msg, last_err) [ 662.832064] env[70557]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 662.832064] env[70557]: ERROR neutron [ 665.676958] env[70784]: INFO neutron.common.config [-] Logging enabled! [ 665.677254] env[70784]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 665.677508] env[70784]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70784) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 665.677760] env[70784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 665.677829] env[70784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 665.678787] env[70784]: WARNING oslo_policy.policy [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 665.678918] env[70784]: WARNING oslo_policy.policy [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 665.680204] env[70784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 665.698751] env[70784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Session token - acquiring [ 665.699667] env[70784]: DEBUG networking_nsxv3.common.locking [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70784) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 665.699879] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 665.700068] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 665.700394] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 665.715682] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 665.715962] env[70784]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 665.716128] env[70784]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70784) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 667.722103] env[70784]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Session token - acquiring [ 667.722758] env[70784]: DEBUG networking_nsxv3.common.locking [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70784) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 667.722758] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 667.722860] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 667.722994] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 667.732295] env[70784]: DEBUG oslo_concurrency.lockutils [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70784) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 667.732829] env[70784]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 667.732829] env[70784]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70784) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 669.735765] env[70784]: CRITICAL neutron [None req-2a9ff863-275a-4384-984e-c449f090cd51 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 669.735765] env[70784]: ERROR neutron Traceback (most recent call last): [ 669.735765] env[70784]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 669.735765] env[70784]: ERROR neutron sys.exit(main()) [ 669.735765] env[70784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 669.735765] env[70784]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 669.735765] env[70784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 669.735765] env[70784]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 669.735765] env[70784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 669.735765] env[70784]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 669.735765] env[70784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 669.735765] env[70784]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 669.735765] env[70784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 669.735765] env[70784]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 669.735765] env[70784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 669.735765] env[70784]: ERROR neutron response = self.get(path=path, params=params) [ 669.736545] env[70784]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 669.736545] env[70784]: ERROR neutron raise RuntimeError(msg, last_err) [ 669.736545] env[70784]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 669.736545] env[70784]: ERROR neutron [ 672.436147] env[71120]: INFO neutron.common.config [-] Logging enabled! [ 672.436460] env[71120]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 672.436682] env[71120]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71120) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 672.436935] env[71120]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 672.437031] env[71120]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 672.437882] env[71120]: WARNING oslo_policy.policy [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 672.438098] env[71120]: WARNING oslo_policy.policy [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 672.439340] env[71120]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 672.453718] env[71120]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Session token - acquiring [ 672.454490] env[71120]: DEBUG networking_nsxv3.common.locking [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71120) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 672.454704] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 672.454842] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 672.455206] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 672.464126] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 672.464378] env[71120]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 672.464556] env[71120]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71120) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 674.470599] env[71120]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Session token - acquiring [ 674.471055] env[71120]: DEBUG networking_nsxv3.common.locking [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71120) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 674.471259] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 674.471385] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 674.471659] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 674.480881] env[71120]: DEBUG oslo_concurrency.lockutils [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71120) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 674.481191] env[71120]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 674.481389] env[71120]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71120) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 676.487134] env[71120]: CRITICAL neutron [None req-e5e5bea9-f970-4a6e-9782-49c9fd20b12b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 676.487134] env[71120]: ERROR neutron Traceback (most recent call last): [ 676.487134] env[71120]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 676.487134] env[71120]: ERROR neutron sys.exit(main()) [ 676.487134] env[71120]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 676.487134] env[71120]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 676.487134] env[71120]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 676.487134] env[71120]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 676.487134] env[71120]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 676.487134] env[71120]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 676.487134] env[71120]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 676.487134] env[71120]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 676.487134] env[71120]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 676.487134] env[71120]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 676.487134] env[71120]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 676.487134] env[71120]: ERROR neutron response = self.get(path=path, params=params) [ 676.488049] env[71120]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 676.488049] env[71120]: ERROR neutron raise RuntimeError(msg, last_err) [ 676.488049] env[71120]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 676.488049] env[71120]: ERROR neutron [ 679.144626] env[71381]: INFO neutron.common.config [-] Logging enabled! [ 679.144960] env[71381]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 679.145158] env[71381]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71381) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 679.145386] env[71381]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 679.145476] env[71381]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 679.146356] env[71381]: WARNING oslo_policy.policy [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 679.146580] env[71381]: WARNING oslo_policy.policy [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 679.147813] env[71381]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 679.163638] env[71381]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Session token - acquiring [ 679.164477] env[71381]: DEBUG networking_nsxv3.common.locking [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71381) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 679.164677] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 679.164802] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 679.165178] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 679.174648] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 679.174895] env[71381]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 679.175038] env[71381]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71381) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 681.181192] env[71381]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Session token - acquiring [ 681.181522] env[71381]: DEBUG networking_nsxv3.common.locking [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71381) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 681.182046] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 681.182046] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 681.182046] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 681.191083] env[71381]: DEBUG oslo_concurrency.lockutils [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71381) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 681.191312] env[71381]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 681.191444] env[71381]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71381) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 683.192956] env[71381]: CRITICAL neutron [None req-f94f271d-1561-4690-979c-aa2b1f35506b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 683.192956] env[71381]: ERROR neutron Traceback (most recent call last): [ 683.192956] env[71381]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 683.192956] env[71381]: ERROR neutron sys.exit(main()) [ 683.192956] env[71381]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 683.192956] env[71381]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 683.192956] env[71381]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 683.192956] env[71381]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 683.192956] env[71381]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 683.192956] env[71381]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 683.192956] env[71381]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 683.192956] env[71381]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 683.192956] env[71381]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 683.192956] env[71381]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 683.192956] env[71381]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 683.192956] env[71381]: ERROR neutron response = self.get(path=path, params=params) [ 683.193775] env[71381]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 683.193775] env[71381]: ERROR neutron raise RuntimeError(msg, last_err) [ 683.193775] env[71381]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 683.193775] env[71381]: ERROR neutron [ 685.834203] env[71479]: INFO neutron.common.config [-] Logging enabled! [ 685.834638] env[71479]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 685.834729] env[71479]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71479) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 685.834969] env[71479]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 685.835063] env[71479]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 685.835885] env[71479]: WARNING oslo_policy.policy [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 685.836167] env[71479]: WARNING oslo_policy.policy [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 685.837363] env[71479]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 685.851664] env[71479]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Session token - acquiring [ 685.852475] env[71479]: DEBUG networking_nsxv3.common.locking [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71479) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 685.852682] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 685.852809] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 685.853166] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 685.861966] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 685.862213] env[71479]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 685.862363] env[71479]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71479) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 687.868266] env[71479]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Session token - acquiring [ 687.868622] env[71479]: DEBUG networking_nsxv3.common.locking [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71479) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 687.868778] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 687.868904] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 687.869182] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 687.878062] env[71479]: DEBUG oslo_concurrency.lockutils [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71479) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 687.878303] env[71479]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 687.878439] env[71479]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71479) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 689.880952] env[71479]: CRITICAL neutron [None req-2c189aa6-bf03-465b-8149-b22487f25f12 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 689.880952] env[71479]: ERROR neutron Traceback (most recent call last): [ 689.880952] env[71479]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 689.880952] env[71479]: ERROR neutron sys.exit(main()) [ 689.880952] env[71479]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 689.880952] env[71479]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 689.880952] env[71479]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 689.880952] env[71479]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 689.880952] env[71479]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 689.880952] env[71479]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 689.880952] env[71479]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 689.880952] env[71479]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 689.880952] env[71479]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 689.880952] env[71479]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 689.880952] env[71479]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 689.880952] env[71479]: ERROR neutron response = self.get(path=path, params=params) [ 689.881711] env[71479]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 689.881711] env[71479]: ERROR neutron raise RuntimeError(msg, last_err) [ 689.881711] env[71479]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 689.881711] env[71479]: ERROR neutron [ 692.675749] env[71539]: INFO neutron.common.config [-] Logging enabled! [ 692.676078] env[71539]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 692.676293] env[71539]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71539) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 692.676560] env[71539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 692.676595] env[71539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 692.677464] env[71539]: WARNING oslo_policy.policy [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 692.677672] env[71539]: WARNING oslo_policy.policy [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 692.678887] env[71539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 692.693967] env[71539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Session token - acquiring [ 692.694883] env[71539]: DEBUG networking_nsxv3.common.locking [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71539) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 692.695146] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 692.695235] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 692.695603] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 692.705539] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 692.705782] env[71539]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 692.705942] env[71539]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71539) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 694.713836] env[71539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Session token - acquiring [ 694.714205] env[71539]: DEBUG networking_nsxv3.common.locking [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71539) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 694.714826] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 694.714826] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 694.714826] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 694.724808] env[71539]: DEBUG oslo_concurrency.lockutils [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 694.725053] env[71539]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 694.725188] env[71539]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71539) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 696.728166] env[71539]: CRITICAL neutron [None req-b9fd5036-d646-4fe4-ae05-8607ecc29220 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 696.728166] env[71539]: ERROR neutron Traceback (most recent call last): [ 696.728166] env[71539]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 696.728166] env[71539]: ERROR neutron sys.exit(main()) [ 696.728166] env[71539]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 696.728166] env[71539]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 696.728166] env[71539]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 696.728166] env[71539]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 696.728166] env[71539]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 696.728166] env[71539]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 696.728166] env[71539]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 696.728166] env[71539]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 696.728166] env[71539]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 696.728166] env[71539]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 696.728166] env[71539]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 696.728166] env[71539]: ERROR neutron response = self.get(path=path, params=params) [ 696.728918] env[71539]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 696.728918] env[71539]: ERROR neutron raise RuntimeError(msg, last_err) [ 696.728918] env[71539]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 696.728918] env[71539]: ERROR neutron [ 699.385810] env[71720]: INFO neutron.common.config [-] Logging enabled! [ 699.386117] env[71720]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 699.386344] env[71720]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71720) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 699.386564] env[71720]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 699.386658] env[71720]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 699.387502] env[71720]: WARNING oslo_policy.policy [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 699.387716] env[71720]: WARNING oslo_policy.policy [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 699.389022] env[71720]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 699.403492] env[71720]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Session token - acquiring [ 699.404247] env[71720]: DEBUG networking_nsxv3.common.locking [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71720) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 699.404451] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 699.404581] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 699.404942] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 699.415228] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 699.415471] env[71720]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 699.415617] env[71720]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71720) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 701.421797] env[71720]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Session token - acquiring [ 701.422645] env[71720]: DEBUG networking_nsxv3.common.locking [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71720) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 701.422645] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 701.422792] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 701.422792] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 701.432495] env[71720]: DEBUG oslo_concurrency.lockutils [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71720) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 701.432753] env[71720]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 701.432892] env[71720]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71720) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 703.435784] env[71720]: CRITICAL neutron [None req-6faf2530-65df-4e6b-884f-9320ced77db1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 703.435784] env[71720]: ERROR neutron Traceback (most recent call last): [ 703.435784] env[71720]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 703.435784] env[71720]: ERROR neutron sys.exit(main()) [ 703.435784] env[71720]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 703.435784] env[71720]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 703.435784] env[71720]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 703.435784] env[71720]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 703.435784] env[71720]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 703.435784] env[71720]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 703.435784] env[71720]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 703.435784] env[71720]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 703.435784] env[71720]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 703.435784] env[71720]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 703.435784] env[71720]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 703.435784] env[71720]: ERROR neutron response = self.get(path=path, params=params) [ 703.436538] env[71720]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 703.436538] env[71720]: ERROR neutron raise RuntimeError(msg, last_err) [ 703.436538] env[71720]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 703.436538] env[71720]: ERROR neutron [ 706.148148] env[71844]: INFO neutron.common.config [-] Logging enabled! [ 706.148148] env[71844]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 706.148148] env[71844]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71844) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 706.148148] env[71844]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 706.148148] env[71844]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 706.148148] env[71844]: WARNING oslo_policy.policy [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 706.148148] env[71844]: WARNING oslo_policy.policy [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 706.148148] env[71844]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 706.161291] env[71844]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Session token - acquiring [ 706.162188] env[71844]: DEBUG networking_nsxv3.common.locking [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71844) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 706.162408] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 706.162530] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 706.162954] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 706.172390] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 706.172648] env[71844]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 706.172801] env[71844]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71844) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 708.178988] env[71844]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Session token - acquiring [ 708.179423] env[71844]: DEBUG networking_nsxv3.common.locking [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71844) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 708.179642] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 708.179772] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 708.180068] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 708.189123] env[71844]: DEBUG oslo_concurrency.lockutils [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71844) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 708.189454] env[71844]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 708.189642] env[71844]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71844) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 710.192557] env[71844]: CRITICAL neutron [None req-c6795b68-93bf-42c6-afeb-b34902f569fb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 710.192557] env[71844]: ERROR neutron Traceback (most recent call last): [ 710.192557] env[71844]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 710.192557] env[71844]: ERROR neutron sys.exit(main()) [ 710.192557] env[71844]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 710.192557] env[71844]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 710.192557] env[71844]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 710.192557] env[71844]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 710.192557] env[71844]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 710.192557] env[71844]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 710.192557] env[71844]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 710.192557] env[71844]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 710.192557] env[71844]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 710.192557] env[71844]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 710.192557] env[71844]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 710.192557] env[71844]: ERROR neutron response = self.get(path=path, params=params) [ 710.193397] env[71844]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 710.193397] env[71844]: ERROR neutron raise RuntimeError(msg, last_err) [ 710.193397] env[71844]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 710.193397] env[71844]: ERROR neutron [ 712.879129] env[71972]: INFO neutron.common.config [-] Logging enabled! [ 712.879484] env[71972]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 712.879673] env[71972]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71972) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 712.879904] env[71972]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 712.879996] env[71972]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 712.880863] env[71972]: WARNING oslo_policy.policy [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 712.881088] env[71972]: WARNING oslo_policy.policy [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 712.882309] env[71972]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 712.896249] env[71972]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Session token - acquiring [ 712.897736] env[71972]: DEBUG networking_nsxv3.common.locking [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71972) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 712.897736] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 712.897898] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 712.897898] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 712.907485] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 712.907763] env[71972]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 712.907918] env[71972]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71972) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 714.915970] env[71972]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Session token - acquiring [ 714.916293] env[71972]: DEBUG networking_nsxv3.common.locking [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71972) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 714.916484] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 714.916642] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 714.916918] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 714.925743] env[71972]: DEBUG oslo_concurrency.lockutils [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71972) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 714.925981] env[71972]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 714.926119] env[71972]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71972) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 716.928934] env[71972]: CRITICAL neutron [None req-ca94bd01-9162-40e1-8d23-dc944157dc93 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 716.928934] env[71972]: ERROR neutron Traceback (most recent call last): [ 716.928934] env[71972]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 716.928934] env[71972]: ERROR neutron sys.exit(main()) [ 716.928934] env[71972]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 716.928934] env[71972]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 716.928934] env[71972]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 716.928934] env[71972]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 716.928934] env[71972]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 716.928934] env[71972]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 716.928934] env[71972]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 716.928934] env[71972]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 716.928934] env[71972]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 716.928934] env[71972]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 716.928934] env[71972]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 716.928934] env[71972]: ERROR neutron response = self.get(path=path, params=params) [ 716.929935] env[71972]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 716.929935] env[71972]: ERROR neutron raise RuntimeError(msg, last_err) [ 716.929935] env[71972]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 716.929935] env[71972]: ERROR neutron [ 719.677140] env[72087]: INFO neutron.common.config [-] Logging enabled! [ 719.677140] env[72087]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 719.677709] env[72087]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72087) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 719.677994] env[72087]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 719.677994] env[72087]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 719.678736] env[72087]: WARNING oslo_policy.policy [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 719.678948] env[72087]: WARNING oslo_policy.policy [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 719.680497] env[72087]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 719.695127] env[72087]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Session token - acquiring [ 719.696268] env[72087]: DEBUG networking_nsxv3.common.locking [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72087) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 719.696520] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 719.696994] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 719.697266] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 719.707618] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 719.707618] env[72087]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 719.707618] env[72087]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72087) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 721.713503] env[72087]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Session token - acquiring [ 721.713858] env[72087]: DEBUG networking_nsxv3.common.locking [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72087) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 721.714037] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 721.714171] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 721.714445] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 721.723511] env[72087]: DEBUG oslo_concurrency.lockutils [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72087) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 721.723859] env[72087]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 721.723949] env[72087]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72087) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 723.724969] env[72087]: CRITICAL neutron [None req-25fae1f2-7855-4e2c-9149-4b534cf58d8d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 723.724969] env[72087]: ERROR neutron Traceback (most recent call last): [ 723.724969] env[72087]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 723.724969] env[72087]: ERROR neutron sys.exit(main()) [ 723.724969] env[72087]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 723.724969] env[72087]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 723.724969] env[72087]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 723.724969] env[72087]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 723.724969] env[72087]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 723.724969] env[72087]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 723.724969] env[72087]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 723.724969] env[72087]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 723.724969] env[72087]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 723.724969] env[72087]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 723.724969] env[72087]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 723.724969] env[72087]: ERROR neutron response = self.get(path=path, params=params) [ 723.725665] env[72087]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 723.725665] env[72087]: ERROR neutron raise RuntimeError(msg, last_err) [ 723.725665] env[72087]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 723.725665] env[72087]: ERROR neutron [ 726.373880] env[72308]: INFO neutron.common.config [-] Logging enabled! [ 726.374216] env[72308]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 726.374429] env[72308]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72308) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 726.374731] env[72308]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 726.374789] env[72308]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 726.375676] env[72308]: WARNING oslo_policy.policy [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 726.375912] env[72308]: WARNING oslo_policy.policy [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 726.377275] env[72308]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 726.398282] env[72308]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Session token - acquiring [ 726.399136] env[72308]: DEBUG networking_nsxv3.common.locking [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72308) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 726.399367] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 726.399491] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 726.399863] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 726.409637] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 726.409935] env[72308]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 726.410091] env[72308]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72308) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 728.416195] env[72308]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Session token - acquiring [ 728.416518] env[72308]: DEBUG networking_nsxv3.common.locking [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72308) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 728.416674] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 728.416796] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 728.417085] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 728.427000] env[72308]: DEBUG oslo_concurrency.lockutils [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72308) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 728.427303] env[72308]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 728.427447] env[72308]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72308) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 730.428956] env[72308]: CRITICAL neutron [None req-cab4a4f7-5f7d-4a2a-980f-48d4fcfe0ef7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 730.428956] env[72308]: ERROR neutron Traceback (most recent call last): [ 730.428956] env[72308]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 730.428956] env[72308]: ERROR neutron sys.exit(main()) [ 730.428956] env[72308]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 730.428956] env[72308]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 730.428956] env[72308]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 730.428956] env[72308]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 730.428956] env[72308]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 730.428956] env[72308]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 730.428956] env[72308]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 730.428956] env[72308]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 730.428956] env[72308]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 730.428956] env[72308]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 730.428956] env[72308]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 730.428956] env[72308]: ERROR neutron response = self.get(path=path, params=params) [ 730.429783] env[72308]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 730.429783] env[72308]: ERROR neutron raise RuntimeError(msg, last_err) [ 730.429783] env[72308]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 730.429783] env[72308]: ERROR neutron [ 733.045090] env[72571]: INFO neutron.common.config [-] Logging enabled! [ 733.045446] env[72571]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 733.045595] env[72571]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72571) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 733.045848] env[72571]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 733.045938] env[72571]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 733.046788] env[72571]: WARNING oslo_policy.policy [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 733.046993] env[72571]: WARNING oslo_policy.policy [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 733.048225] env[72571]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 733.071340] env[72571]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Session token - acquiring [ 733.071340] env[72571]: DEBUG networking_nsxv3.common.locking [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72571) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 733.071340] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 733.071876] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 733.071876] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 733.073236] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 733.073494] env[72571]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 733.073642] env[72571]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72571) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 735.080148] env[72571]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Session token - acquiring [ 735.080421] env[72571]: DEBUG networking_nsxv3.common.locking [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72571) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 735.080517] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 735.080647] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 735.080925] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 735.089803] env[72571]: DEBUG oslo_concurrency.lockutils [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72571) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 735.090055] env[72571]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 735.090195] env[72571]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72571) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 737.092891] env[72571]: CRITICAL neutron [None req-526c22eb-e922-437c-9748-b6047edd4c9c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 737.092891] env[72571]: ERROR neutron Traceback (most recent call last): [ 737.092891] env[72571]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 737.092891] env[72571]: ERROR neutron sys.exit(main()) [ 737.092891] env[72571]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 737.092891] env[72571]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 737.092891] env[72571]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 737.092891] env[72571]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 737.092891] env[72571]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 737.092891] env[72571]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 737.092891] env[72571]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 737.092891] env[72571]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 737.092891] env[72571]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 737.092891] env[72571]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 737.092891] env[72571]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 737.092891] env[72571]: ERROR neutron response = self.get(path=path, params=params) [ 737.093663] env[72571]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 737.093663] env[72571]: ERROR neutron raise RuntimeError(msg, last_err) [ 737.093663] env[72571]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 737.093663] env[72571]: ERROR neutron [ 739.710130] env[72837]: INFO neutron.common.config [-] Logging enabled! [ 739.710509] env[72837]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 739.710689] env[72837]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72837) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 739.710936] env[72837]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 739.711026] env[72837]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 739.711880] env[72837]: WARNING oslo_policy.policy [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 739.712195] env[72837]: WARNING oslo_policy.policy [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 739.713471] env[72837]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 739.728064] env[72837]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Session token - acquiring [ 739.728917] env[72837]: DEBUG networking_nsxv3.common.locking [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72837) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 739.729168] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 739.729243] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 739.729662] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 739.738800] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 739.739055] env[72837]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 739.739201] env[72837]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72837) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 741.747584] env[72837]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Session token - acquiring [ 741.747584] env[72837]: DEBUG networking_nsxv3.common.locking [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72837) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 741.747584] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 741.748224] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 741.748224] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 741.755997] env[72837]: DEBUG oslo_concurrency.lockutils [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72837) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 741.756459] env[72837]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 741.756754] env[72837]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72837) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 743.758317] env[72837]: CRITICAL neutron [None req-9aa207bc-657a-4140-8c52-1bdd7a9bff12 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 743.758317] env[72837]: ERROR neutron Traceback (most recent call last): [ 743.758317] env[72837]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 743.758317] env[72837]: ERROR neutron sys.exit(main()) [ 743.758317] env[72837]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 743.758317] env[72837]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 743.758317] env[72837]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 743.758317] env[72837]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 743.758317] env[72837]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 743.758317] env[72837]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 743.758317] env[72837]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 743.758317] env[72837]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 743.758317] env[72837]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 743.758317] env[72837]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 743.758317] env[72837]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 743.758317] env[72837]: ERROR neutron response = self.get(path=path, params=params) [ 743.759062] env[72837]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 743.759062] env[72837]: ERROR neutron raise RuntimeError(msg, last_err) [ 743.759062] env[72837]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 743.759062] env[72837]: ERROR neutron [ 746.493208] env[72993]: INFO neutron.common.config [-] Logging enabled! [ 746.493473] env[72993]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 746.493757] env[72993]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72993) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 746.493997] env[72993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 746.494058] env[72993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 746.495004] env[72993]: WARNING oslo_policy.policy [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 746.495190] env[72993]: WARNING oslo_policy.policy [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 746.496406] env[72993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 746.510927] env[72993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Session token - acquiring [ 746.511807] env[72993]: DEBUG networking_nsxv3.common.locking [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72993) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 746.512016] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 746.512201] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 746.512656] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 746.522276] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 746.522276] env[72993]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 746.522404] env[72993]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72993) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 748.529266] env[72993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Session token - acquiring [ 748.531761] env[72993]: DEBUG networking_nsxv3.common.locking [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72993) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 748.531761] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 748.532205] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 748.532205] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 748.542711] env[72993]: DEBUG oslo_concurrency.lockutils [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 748.542711] env[72993]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 748.542711] env[72993]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72993) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 750.544873] env[72993]: CRITICAL neutron [None req-a6eb73ed-2601-43d7-9512-338b1278df7a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 750.544873] env[72993]: ERROR neutron Traceback (most recent call last): [ 750.544873] env[72993]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 750.544873] env[72993]: ERROR neutron sys.exit(main()) [ 750.544873] env[72993]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 750.544873] env[72993]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 750.544873] env[72993]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 750.544873] env[72993]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 750.544873] env[72993]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 750.544873] env[72993]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 750.544873] env[72993]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 750.544873] env[72993]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 750.544873] env[72993]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 750.544873] env[72993]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 750.544873] env[72993]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 750.544873] env[72993]: ERROR neutron response = self.get(path=path, params=params) [ 750.545579] env[72993]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 750.545579] env[72993]: ERROR neutron raise RuntimeError(msg, last_err) [ 750.545579] env[72993]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 750.545579] env[72993]: ERROR neutron [ 753.066262] env[73294]: INFO neutron.common.config [-] Logging enabled! [ 753.066262] env[73294]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 753.066262] env[73294]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73294) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 753.066262] env[73294]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 753.066262] env[73294]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 753.066702] env[73294]: WARNING oslo_policy.policy [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 753.067031] env[73294]: WARNING oslo_policy.policy [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 753.068549] env[73294]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 753.082539] env[73294]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Session token - acquiring [ 753.083365] env[73294]: DEBUG networking_nsxv3.common.locking [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73294) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 753.083570] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 753.083695] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 753.084075] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 753.092989] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 753.093239] env[73294]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 753.093387] env[73294]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73294) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 755.099407] env[73294]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Session token - acquiring [ 755.099707] env[73294]: DEBUG networking_nsxv3.common.locking [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73294) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 755.099839] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 755.099960] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 755.100256] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 755.110168] env[73294]: DEBUG oslo_concurrency.lockutils [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73294) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 755.110448] env[73294]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 755.110589] env[73294]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73294) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 757.113018] env[73294]: CRITICAL neutron [None req-388c20ec-1ae4-4433-9056-f82287e86c0b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 757.113018] env[73294]: ERROR neutron Traceback (most recent call last): [ 757.113018] env[73294]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 757.113018] env[73294]: ERROR neutron sys.exit(main()) [ 757.113018] env[73294]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 757.113018] env[73294]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 757.113018] env[73294]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 757.113018] env[73294]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 757.113018] env[73294]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 757.113018] env[73294]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 757.113018] env[73294]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 757.113018] env[73294]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 757.113018] env[73294]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 757.113018] env[73294]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 757.113018] env[73294]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 757.113018] env[73294]: ERROR neutron response = self.get(path=path, params=params) [ 757.114286] env[73294]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 757.114286] env[73294]: ERROR neutron raise RuntimeError(msg, last_err) [ 757.114286] env[73294]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 757.114286] env[73294]: ERROR neutron [ 759.613821] env[73414]: INFO neutron.common.config [-] Logging enabled! [ 759.614164] env[73414]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 759.614351] env[73414]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73414) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 759.614642] env[73414]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 759.614728] env[73414]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 759.615583] env[73414]: WARNING oslo_policy.policy [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 759.615790] env[73414]: WARNING oslo_policy.policy [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 759.617391] env[73414]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 759.631484] env[73414]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Session token - acquiring [ 759.632558] env[73414]: DEBUG networking_nsxv3.common.locking [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73414) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 759.632801] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 759.632928] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 759.633299] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 759.642557] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 759.642854] env[73414]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 759.643020] env[73414]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73414) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 761.650131] env[73414]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Session token - acquiring [ 761.650447] env[73414]: DEBUG networking_nsxv3.common.locking [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73414) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 761.650519] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 761.650698] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 761.651080] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 761.661816] env[73414]: DEBUG oslo_concurrency.lockutils [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73414) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 761.662058] env[73414]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 761.662196] env[73414]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73414) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 763.664948] env[73414]: CRITICAL neutron [None req-0036a574-4d37-40d3-bd3e-04c374e8481d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 763.664948] env[73414]: ERROR neutron Traceback (most recent call last): [ 763.664948] env[73414]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 763.664948] env[73414]: ERROR neutron sys.exit(main()) [ 763.664948] env[73414]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 763.664948] env[73414]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 763.664948] env[73414]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 763.664948] env[73414]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 763.664948] env[73414]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 763.664948] env[73414]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 763.664948] env[73414]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 763.664948] env[73414]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 763.664948] env[73414]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 763.664948] env[73414]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 763.664948] env[73414]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 763.664948] env[73414]: ERROR neutron response = self.get(path=path, params=params) [ 763.665643] env[73414]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 763.665643] env[73414]: ERROR neutron raise RuntimeError(msg, last_err) [ 763.665643] env[73414]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 763.665643] env[73414]: ERROR neutron [ 766.467582] env[73543]: INFO neutron.common.config [-] Logging enabled! [ 766.467582] env[73543]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 766.467975] env[73543]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73543) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 766.468238] env[73543]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 766.468337] env[73543]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 766.469182] env[73543]: WARNING oslo_policy.policy [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 766.469424] env[73543]: WARNING oslo_policy.policy [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 766.470681] env[73543]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 766.485219] env[73543]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Session token - acquiring [ 766.486010] env[73543]: DEBUG networking_nsxv3.common.locking [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73543) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 766.486210] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 766.486333] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 766.486698] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 766.495791] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 766.496063] env[73543]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 766.496215] env[73543]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73543) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 768.502670] env[73543]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Session token - acquiring [ 768.502953] env[73543]: DEBUG networking_nsxv3.common.locking [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73543) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 768.503141] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 768.503260] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 768.503615] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 768.519963] env[73543]: DEBUG oslo_concurrency.lockutils [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73543) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 768.520164] env[73543]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 768.520278] env[73543]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73543) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 770.523273] env[73543]: CRITICAL neutron [None req-0dbd54f3-6186-4cf6-ab3f-877b608cc33b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 770.523273] env[73543]: ERROR neutron Traceback (most recent call last): [ 770.523273] env[73543]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 770.523273] env[73543]: ERROR neutron sys.exit(main()) [ 770.523273] env[73543]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 770.523273] env[73543]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 770.523273] env[73543]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 770.523273] env[73543]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 770.523273] env[73543]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 770.523273] env[73543]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 770.523273] env[73543]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 770.523273] env[73543]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 770.523273] env[73543]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 770.523273] env[73543]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 770.523273] env[73543]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 770.523273] env[73543]: ERROR neutron response = self.get(path=path, params=params) [ 770.524046] env[73543]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 770.524046] env[73543]: ERROR neutron raise RuntimeError(msg, last_err) [ 770.524046] env[73543]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 770.524046] env[73543]: ERROR neutron [ 773.180822] env[73678]: INFO neutron.common.config [-] Logging enabled! [ 773.181180] env[73678]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 773.181379] env[73678]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73678) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 773.181598] env[73678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 773.181704] env[73678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 773.182587] env[73678]: WARNING oslo_policy.policy [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 773.182822] env[73678]: WARNING oslo_policy.policy [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 773.184095] env[73678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 773.207733] env[73678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Session token - acquiring [ 773.208675] env[73678]: DEBUG networking_nsxv3.common.locking [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73678) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 773.208886] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 773.209009] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 773.209415] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 773.224912] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 773.225187] env[73678]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 773.225376] env[73678]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73678) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 775.232174] env[73678]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Session token - acquiring [ 775.232174] env[73678]: DEBUG networking_nsxv3.common.locking [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73678) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 775.232174] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 775.232477] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 775.232477] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 775.241512] env[73678]: DEBUG oslo_concurrency.lockutils [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73678) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 775.241755] env[73678]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 775.241894] env[73678]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73678) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 777.243447] env[73678]: CRITICAL neutron [None req-fc48bba0-bd2c-498d-bd93-dadaefd5786c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 777.243447] env[73678]: ERROR neutron Traceback (most recent call last): [ 777.243447] env[73678]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 777.243447] env[73678]: ERROR neutron sys.exit(main()) [ 777.243447] env[73678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 777.243447] env[73678]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 777.243447] env[73678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 777.243447] env[73678]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 777.243447] env[73678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 777.243447] env[73678]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 777.243447] env[73678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 777.243447] env[73678]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 777.243447] env[73678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 777.243447] env[73678]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 777.243447] env[73678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 777.243447] env[73678]: ERROR neutron response = self.get(path=path, params=params) [ 777.244238] env[73678]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 777.244238] env[73678]: ERROR neutron raise RuntimeError(msg, last_err) [ 777.244238] env[73678]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 777.244238] env[73678]: ERROR neutron [ 779.921242] env[73794]: INFO neutron.common.config [-] Logging enabled! [ 779.921583] env[73794]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 779.921778] env[73794]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73794) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 779.922021] env[73794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 779.922116] env[73794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 779.922937] env[73794]: WARNING oslo_policy.policy [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 779.923157] env[73794]: WARNING oslo_policy.policy [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 779.924405] env[73794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 779.938591] env[73794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Session token - acquiring [ 779.939385] env[73794]: DEBUG networking_nsxv3.common.locking [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73794) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 779.939584] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 779.939712] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 779.940097] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 779.953290] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 779.953552] env[73794]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 779.953703] env[73794]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73794) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 781.960813] env[73794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Session token - acquiring [ 781.961110] env[73794]: DEBUG networking_nsxv3.common.locking [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73794) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 781.961172] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 781.961321] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 781.961669] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 781.970843] env[73794]: DEBUG oslo_concurrency.lockutils [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73794) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 781.971043] env[73794]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 781.971181] env[73794]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73794) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 783.973092] env[73794]: CRITICAL neutron [None req-094139a2-f58c-4f7f-9c7a-e34ac3d4b7d7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 783.973092] env[73794]: ERROR neutron Traceback (most recent call last): [ 783.973092] env[73794]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 783.973092] env[73794]: ERROR neutron sys.exit(main()) [ 783.973092] env[73794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 783.973092] env[73794]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 783.973092] env[73794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 783.973092] env[73794]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 783.973092] env[73794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 783.973092] env[73794]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 783.973092] env[73794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 783.973092] env[73794]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 783.973092] env[73794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 783.973092] env[73794]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 783.973092] env[73794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 783.973092] env[73794]: ERROR neutron response = self.get(path=path, params=params) [ 783.973788] env[73794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 783.973788] env[73794]: ERROR neutron raise RuntimeError(msg, last_err) [ 783.973788] env[73794]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 783.973788] env[73794]: ERROR neutron [ 786.658043] env[73911]: INFO neutron.common.config [-] Logging enabled! [ 786.658328] env[73911]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 786.658569] env[73911]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73911) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 786.658798] env[73911]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 786.658979] env[73911]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 786.659868] env[73911]: WARNING oslo_policy.policy [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 786.660138] env[73911]: WARNING oslo_policy.policy [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 786.662245] env[73911]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 786.683995] env[73911]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Session token - acquiring [ 786.684853] env[73911]: DEBUG networking_nsxv3.common.locking [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73911) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 786.685062] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 786.685188] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 786.685547] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 786.695179] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 786.695437] env[73911]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 786.695588] env[73911]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73911) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 788.702046] env[73911]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Session token - acquiring [ 788.702046] env[73911]: DEBUG networking_nsxv3.common.locking [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73911) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 788.702366] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 788.702366] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 788.702634] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 788.711744] env[73911]: DEBUG oslo_concurrency.lockutils [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73911) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 788.711992] env[73911]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 788.712152] env[73911]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73911) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 790.715217] env[73911]: CRITICAL neutron [None req-0986d058-6e5c-416f-bae9-ac73f8b2fcba None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 790.715217] env[73911]: ERROR neutron Traceback (most recent call last): [ 790.715217] env[73911]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 790.715217] env[73911]: ERROR neutron sys.exit(main()) [ 790.715217] env[73911]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 790.715217] env[73911]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 790.715217] env[73911]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 790.715217] env[73911]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 790.715217] env[73911]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 790.715217] env[73911]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 790.715217] env[73911]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 790.715217] env[73911]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 790.715217] env[73911]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 790.715217] env[73911]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 790.715217] env[73911]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 790.715217] env[73911]: ERROR neutron response = self.get(path=path, params=params) [ 790.715944] env[73911]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 790.715944] env[73911]: ERROR neutron raise RuntimeError(msg, last_err) [ 790.715944] env[73911]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 790.715944] env[73911]: ERROR neutron [ 793.424534] env[74029]: INFO neutron.common.config [-] Logging enabled! [ 793.424875] env[74029]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 793.425070] env[74029]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74029) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 793.425296] env[74029]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 793.425404] env[74029]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 793.426221] env[74029]: WARNING oslo_policy.policy [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 793.426443] env[74029]: WARNING oslo_policy.policy [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 793.427734] env[74029]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 793.452167] env[74029]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Session token - acquiring [ 793.453114] env[74029]: DEBUG networking_nsxv3.common.locking [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74029) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 793.453319] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 793.453444] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 793.453829] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 793.463453] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 793.463706] env[74029]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 793.463858] env[74029]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74029) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 795.469886] env[74029]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Session token - acquiring [ 795.470199] env[74029]: DEBUG networking_nsxv3.common.locking [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74029) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 795.470391] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 795.470511] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 795.470786] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 795.480398] env[74029]: DEBUG oslo_concurrency.lockutils [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74029) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 795.480628] env[74029]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 795.480764] env[74029]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74029) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 797.483660] env[74029]: CRITICAL neutron [None req-fb7fc6e1-cc79-474d-9e5c-1920de4b19e2 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 797.483660] env[74029]: ERROR neutron Traceback (most recent call last): [ 797.483660] env[74029]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 797.483660] env[74029]: ERROR neutron sys.exit(main()) [ 797.483660] env[74029]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 797.483660] env[74029]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 797.483660] env[74029]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 797.483660] env[74029]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 797.483660] env[74029]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 797.483660] env[74029]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 797.483660] env[74029]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 797.483660] env[74029]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 797.483660] env[74029]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 797.483660] env[74029]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 797.483660] env[74029]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 797.483660] env[74029]: ERROR neutron response = self.get(path=path, params=params) [ 797.484345] env[74029]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 797.484345] env[74029]: ERROR neutron raise RuntimeError(msg, last_err) [ 797.484345] env[74029]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 797.484345] env[74029]: ERROR neutron [ 800.090226] env[74224]: INFO neutron.common.config [-] Logging enabled! [ 800.090503] env[74224]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 800.090951] env[74224]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74224) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 800.091025] env[74224]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 800.091086] env[74224]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 800.091908] env[74224]: WARNING oslo_policy.policy [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 800.092143] env[74224]: WARNING oslo_policy.policy [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 800.093387] env[74224]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 800.107976] env[74224]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Session token - acquiring [ 800.108828] env[74224]: DEBUG networking_nsxv3.common.locking [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74224) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 800.109027] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 800.109155] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 800.109700] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 800.121535] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 800.121822] env[74224]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 800.121971] env[74224]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74224) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 802.130082] env[74224]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Session token - acquiring [ 802.130502] env[74224]: DEBUG networking_nsxv3.common.locking [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74224) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 802.130727] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 802.130851] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 802.131123] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 802.140233] env[74224]: DEBUG oslo_concurrency.lockutils [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74224) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 802.140510] env[74224]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 802.140679] env[74224]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74224) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 804.143597] env[74224]: CRITICAL neutron [None req-bf535b95-f4e3-44fe-8ddb-953afffec809 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 804.143597] env[74224]: ERROR neutron Traceback (most recent call last): [ 804.143597] env[74224]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 804.143597] env[74224]: ERROR neutron sys.exit(main()) [ 804.143597] env[74224]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 804.143597] env[74224]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 804.143597] env[74224]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 804.143597] env[74224]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 804.143597] env[74224]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 804.143597] env[74224]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 804.143597] env[74224]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 804.143597] env[74224]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 804.143597] env[74224]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 804.143597] env[74224]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 804.143597] env[74224]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 804.143597] env[74224]: ERROR neutron response = self.get(path=path, params=params) [ 804.144336] env[74224]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 804.144336] env[74224]: ERROR neutron raise RuntimeError(msg, last_err) [ 804.144336] env[74224]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 804.144336] env[74224]: ERROR neutron [ 806.925248] env[74337]: INFO neutron.common.config [-] Logging enabled! [ 806.925581] env[74337]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 806.925762] env[74337]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74337) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 806.925982] env[74337]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 806.926072] env[74337]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 806.926879] env[74337]: WARNING oslo_policy.policy [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 806.927092] env[74337]: WARNING oslo_policy.policy [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 806.928265] env[74337]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 806.942228] env[74337]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Session token - acquiring [ 806.942960] env[74337]: DEBUG networking_nsxv3.common.locking [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74337) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 806.943155] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 806.943280] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 806.943629] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 806.952603] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 806.952844] env[74337]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 806.952991] env[74337]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74337) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 808.958967] env[74337]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Session token - acquiring [ 808.959433] env[74337]: DEBUG networking_nsxv3.common.locking [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74337) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 808.959595] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 808.959730] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 808.960020] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 808.969795] env[74337]: DEBUG oslo_concurrency.lockutils [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74337) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 808.970039] env[74337]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 808.970227] env[74337]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74337) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 810.972943] env[74337]: CRITICAL neutron [None req-d5284908-5aef-4f04-9dbd-32aa30ff369c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 810.972943] env[74337]: ERROR neutron Traceback (most recent call last): [ 810.972943] env[74337]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 810.972943] env[74337]: ERROR neutron sys.exit(main()) [ 810.972943] env[74337]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 810.972943] env[74337]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 810.972943] env[74337]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 810.972943] env[74337]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 810.972943] env[74337]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 810.972943] env[74337]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 810.972943] env[74337]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 810.972943] env[74337]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 810.972943] env[74337]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 810.972943] env[74337]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 810.972943] env[74337]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 810.972943] env[74337]: ERROR neutron response = self.get(path=path, params=params) [ 810.973500] env[74337]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 810.973500] env[74337]: ERROR neutron raise RuntimeError(msg, last_err) [ 810.973500] env[74337]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 810.973500] env[74337]: ERROR neutron [ 813.756660] env[74443]: INFO neutron.common.config [-] Logging enabled! [ 813.756922] env[74443]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 813.757181] env[74443]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74443) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 813.757445] env[74443]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 813.757547] env[74443]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 813.758383] env[74443]: WARNING oslo_policy.policy [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 813.758601] env[74443]: WARNING oslo_policy.policy [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 813.759818] env[74443]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 813.773867] env[74443]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Session token - acquiring [ 813.774694] env[74443]: DEBUG networking_nsxv3.common.locking [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74443) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 813.774903] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 813.775029] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 813.775394] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 813.785167] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 813.785401] env[74443]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 813.785546] env[74443]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74443) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 815.792530] env[74443]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Session token - acquiring [ 815.792890] env[74443]: DEBUG networking_nsxv3.common.locking [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74443) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 815.793105] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 815.793256] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 815.793557] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 815.804852] env[74443]: DEBUG oslo_concurrency.lockutils [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74443) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 815.805136] env[74443]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 815.805278] env[74443]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74443) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 817.808428] env[74443]: CRITICAL neutron [None req-c94ccd74-62c7-43ac-9d59-3c4a4d248287 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 817.808428] env[74443]: ERROR neutron Traceback (most recent call last): [ 817.808428] env[74443]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 817.808428] env[74443]: ERROR neutron sys.exit(main()) [ 817.808428] env[74443]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 817.808428] env[74443]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 817.808428] env[74443]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 817.808428] env[74443]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 817.808428] env[74443]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 817.808428] env[74443]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 817.808428] env[74443]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 817.808428] env[74443]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 817.808428] env[74443]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 817.808428] env[74443]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 817.808428] env[74443]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 817.808428] env[74443]: ERROR neutron response = self.get(path=path, params=params) [ 817.809051] env[74443]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 817.809051] env[74443]: ERROR neutron raise RuntimeError(msg, last_err) [ 817.809051] env[74443]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 817.809051] env[74443]: ERROR neutron [ 820.240454] env[74791]: INFO neutron.common.config [-] Logging enabled! [ 820.240745] env[74791]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 820.240995] env[74791]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74791) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 820.241240] env[74791]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 820.241356] env[74791]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 820.242200] env[74791]: WARNING oslo_policy.policy [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 820.242415] env[74791]: WARNING oslo_policy.policy [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 820.243667] env[74791]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 820.258489] env[74791]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Session token - acquiring [ 820.259289] env[74791]: DEBUG networking_nsxv3.common.locking [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74791) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 820.259517] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 820.259650] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 820.260018] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 820.269221] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 820.269493] env[74791]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 820.269643] env[74791]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74791) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 822.276054] env[74791]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Session token - acquiring [ 822.277046] env[74791]: DEBUG networking_nsxv3.common.locking [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74791) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 822.277046] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 822.277280] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 822.277280] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 822.286900] env[74791]: DEBUG oslo_concurrency.lockutils [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74791) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 822.286900] env[74791]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 822.286900] env[74791]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74791) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 824.289632] env[74791]: CRITICAL neutron [None req-6eb9e2cd-2aa9-44e3-987c-2bc9a2dde64a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 824.289632] env[74791]: ERROR neutron Traceback (most recent call last): [ 824.289632] env[74791]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 824.289632] env[74791]: ERROR neutron sys.exit(main()) [ 824.289632] env[74791]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 824.289632] env[74791]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 824.289632] env[74791]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 824.289632] env[74791]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 824.289632] env[74791]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 824.289632] env[74791]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 824.289632] env[74791]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 824.289632] env[74791]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 824.289632] env[74791]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 824.289632] env[74791]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 824.289632] env[74791]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 824.289632] env[74791]: ERROR neutron response = self.get(path=path, params=params) [ 824.290333] env[74791]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 824.290333] env[74791]: ERROR neutron raise RuntimeError(msg, last_err) [ 824.290333] env[74791]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 824.290333] env[74791]: ERROR neutron [ 826.899428] env[74895]: INFO neutron.common.config [-] Logging enabled! [ 826.899764] env[74895]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 826.899963] env[74895]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74895) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 826.900341] env[74895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 826.900469] env[74895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 826.901332] env[74895]: WARNING oslo_policy.policy [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 826.901545] env[74895]: WARNING oslo_policy.policy [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 826.902781] env[74895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 826.916996] env[74895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Session token - acquiring [ 826.917872] env[74895]: DEBUG networking_nsxv3.common.locking [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74895) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 826.918078] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 826.918204] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 826.918574] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 826.927752] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 826.927999] env[74895]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 826.928163] env[74895]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74895) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 828.934439] env[74895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Session token - acquiring [ 828.934699] env[74895]: DEBUG networking_nsxv3.common.locking [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74895) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 828.934897] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 828.935021] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 828.935292] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 828.950114] env[74895]: DEBUG oslo_concurrency.lockutils [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 828.950441] env[74895]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 828.950626] env[74895]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74895) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 830.956931] env[74895]: CRITICAL neutron [None req-fe7462ce-1f8c-4337-99fd-d00f1e5526b5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 830.956931] env[74895]: ERROR neutron Traceback (most recent call last): [ 830.956931] env[74895]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 830.956931] env[74895]: ERROR neutron sys.exit(main()) [ 830.956931] env[74895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 830.956931] env[74895]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 830.956931] env[74895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 830.956931] env[74895]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 830.956931] env[74895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 830.956931] env[74895]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 830.956931] env[74895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 830.956931] env[74895]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 830.956931] env[74895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 830.956931] env[74895]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 830.956931] env[74895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 830.956931] env[74895]: ERROR neutron response = self.get(path=path, params=params) [ 830.957939] env[74895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 830.957939] env[74895]: ERROR neutron raise RuntimeError(msg, last_err) [ 830.957939] env[74895]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 830.957939] env[74895]: ERROR neutron [ 833.551046] env[75063]: INFO neutron.common.config [-] Logging enabled! [ 833.551537] env[75063]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 833.551584] env[75063]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75063) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 833.551787] env[75063]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 833.551880] env[75063]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 833.552731] env[75063]: WARNING oslo_policy.policy [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 833.552951] env[75063]: WARNING oslo_policy.policy [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 833.554152] env[75063]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 833.568270] env[75063]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Session token - acquiring [ 833.569022] env[75063]: DEBUG networking_nsxv3.common.locking [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75063) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 833.569225] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 833.569353] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 833.569769] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 833.578346] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 833.578587] env[75063]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 833.578742] env[75063]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75063) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 835.584954] env[75063]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Session token - acquiring [ 835.585325] env[75063]: DEBUG networking_nsxv3.common.locking [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75063) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 835.585569] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 835.585768] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 835.586043] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 835.595171] env[75063]: DEBUG oslo_concurrency.lockutils [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75063) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 835.595401] env[75063]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 835.595540] env[75063]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75063) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 837.596896] env[75063]: CRITICAL neutron [None req-473f9e63-2806-4f84-aa98-e059d43663cb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 837.596896] env[75063]: ERROR neutron Traceback (most recent call last): [ 837.596896] env[75063]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 837.596896] env[75063]: ERROR neutron sys.exit(main()) [ 837.596896] env[75063]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 837.596896] env[75063]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 837.596896] env[75063]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 837.596896] env[75063]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 837.596896] env[75063]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 837.596896] env[75063]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 837.596896] env[75063]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 837.596896] env[75063]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 837.596896] env[75063]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 837.596896] env[75063]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 837.596896] env[75063]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 837.596896] env[75063]: ERROR neutron response = self.get(path=path, params=params) [ 837.597817] env[75063]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 837.597817] env[75063]: ERROR neutron raise RuntimeError(msg, last_err) [ 837.597817] env[75063]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 837.597817] env[75063]: ERROR neutron [ 840.130375] env[75106]: INFO neutron.common.config [-] Logging enabled! [ 840.130788] env[75106]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 840.130999] env[75106]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75106) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 840.131238] env[75106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 840.131335] env[75106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 840.132183] env[75106]: WARNING oslo_policy.policy [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 840.132406] env[75106]: WARNING oslo_policy.policy [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 840.133620] env[75106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 840.148497] env[75106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Session token - acquiring [ 840.149305] env[75106]: DEBUG networking_nsxv3.common.locking [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75106) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 840.149516] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 840.149647] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 840.150013] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 840.159663] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 840.159910] env[75106]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 840.160066] env[75106]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75106) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 842.168057] env[75106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Session token - acquiring [ 842.168057] env[75106]: DEBUG networking_nsxv3.common.locking [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75106) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 842.168057] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 842.168458] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 842.168458] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 842.175849] env[75106]: DEBUG oslo_concurrency.lockutils [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 842.176228] env[75106]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 842.176470] env[75106]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75106) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 844.179492] env[75106]: CRITICAL neutron [None req-8f6c18a5-bedf-4225-a7ef-5001170584c5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 844.179492] env[75106]: ERROR neutron Traceback (most recent call last): [ 844.179492] env[75106]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 844.179492] env[75106]: ERROR neutron sys.exit(main()) [ 844.179492] env[75106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 844.179492] env[75106]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 844.179492] env[75106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 844.179492] env[75106]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 844.179492] env[75106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 844.179492] env[75106]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 844.179492] env[75106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 844.179492] env[75106]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 844.179492] env[75106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 844.179492] env[75106]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 844.179492] env[75106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 844.179492] env[75106]: ERROR neutron response = self.get(path=path, params=params) [ 844.180178] env[75106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 844.180178] env[75106]: ERROR neutron raise RuntimeError(msg, last_err) [ 844.180178] env[75106]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 844.180178] env[75106]: ERROR neutron [ 846.808494] env[75222]: INFO neutron.common.config [-] Logging enabled! [ 846.808783] env[75222]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 846.809042] env[75222]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75222) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 846.809260] env[75222]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 846.809353] env[75222]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 846.810230] env[75222]: WARNING oslo_policy.policy [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 846.810470] env[75222]: WARNING oslo_policy.policy [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 846.811638] env[75222]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 846.831650] env[75222]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Session token - acquiring [ 846.832487] env[75222]: DEBUG networking_nsxv3.common.locking [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75222) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 846.832704] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 846.832833] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 846.833195] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 846.841900] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 846.842147] env[75222]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 846.842302] env[75222]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75222) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 848.848271] env[75222]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Session token - acquiring [ 848.848610] env[75222]: DEBUG networking_nsxv3.common.locking [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75222) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 848.848843] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 848.848969] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 848.849234] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 848.858864] env[75222]: DEBUG oslo_concurrency.lockutils [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75222) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 848.858943] env[75222]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 848.859138] env[75222]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75222) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 850.860927] env[75222]: CRITICAL neutron [None req-4f670b8a-c128-429c-82b6-8c7322d42eb5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 850.860927] env[75222]: ERROR neutron Traceback (most recent call last): [ 850.860927] env[75222]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 850.860927] env[75222]: ERROR neutron sys.exit(main()) [ 850.860927] env[75222]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 850.860927] env[75222]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 850.860927] env[75222]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 850.860927] env[75222]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 850.860927] env[75222]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 850.860927] env[75222]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 850.860927] env[75222]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 850.860927] env[75222]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 850.860927] env[75222]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 850.860927] env[75222]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 850.860927] env[75222]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 850.860927] env[75222]: ERROR neutron response = self.get(path=path, params=params) [ 850.861540] env[75222]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 850.861540] env[75222]: ERROR neutron raise RuntimeError(msg, last_err) [ 850.861540] env[75222]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 850.861540] env[75222]: ERROR neutron [ 853.463271] env[75293]: INFO neutron.common.config [-] Logging enabled! [ 853.463707] env[75293]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 853.463825] env[75293]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75293) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 853.464124] env[75293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 853.464227] env[75293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 853.465108] env[75293]: WARNING oslo_policy.policy [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 853.465343] env[75293]: WARNING oslo_policy.policy [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 853.466588] env[75293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 853.481545] env[75293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Session token - acquiring [ 853.482458] env[75293]: DEBUG networking_nsxv3.common.locking [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75293) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 853.482668] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 853.482794] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 853.483169] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 853.493211] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 853.493531] env[75293]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 853.493651] env[75293]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75293) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 855.499871] env[75293]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Session token - acquiring [ 855.500234] env[75293]: DEBUG networking_nsxv3.common.locking [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75293) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 855.500372] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 855.500547] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 855.500824] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 855.511167] env[75293]: DEBUG oslo_concurrency.lockutils [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75293) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 855.511413] env[75293]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 855.511547] env[75293]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75293) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 857.512968] env[75293]: CRITICAL neutron [None req-927f1060-fe5d-460f-acb3-588ed2f9a464 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 857.512968] env[75293]: ERROR neutron Traceback (most recent call last): [ 857.512968] env[75293]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 857.512968] env[75293]: ERROR neutron sys.exit(main()) [ 857.512968] env[75293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 857.512968] env[75293]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 857.512968] env[75293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 857.512968] env[75293]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 857.512968] env[75293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 857.512968] env[75293]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 857.512968] env[75293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 857.512968] env[75293]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 857.512968] env[75293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 857.512968] env[75293]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 857.512968] env[75293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 857.512968] env[75293]: ERROR neutron response = self.get(path=path, params=params) [ 857.513822] env[75293]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 857.513822] env[75293]: ERROR neutron raise RuntimeError(msg, last_err) [ 857.513822] env[75293]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 857.513822] env[75293]: ERROR neutron [ 860.173379] env[75557]: INFO neutron.common.config [-] Logging enabled! [ 860.173667] env[75557]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 860.173905] env[75557]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75557) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 860.174129] env[75557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 860.174223] env[75557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 860.175076] env[75557]: WARNING oslo_policy.policy [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 860.175296] env[75557]: WARNING oslo_policy.policy [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 860.176505] env[75557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 860.191060] env[75557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Session token - acquiring [ 860.192311] env[75557]: DEBUG networking_nsxv3.common.locking [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75557) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 860.192311] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 860.192460] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 860.192773] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 860.202947] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 860.203197] env[75557]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 860.203400] env[75557]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75557) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 862.209520] env[75557]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Session token - acquiring [ 862.209831] env[75557]: DEBUG networking_nsxv3.common.locking [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75557) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 862.209831] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 862.209948] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 862.210222] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 862.219103] env[75557]: DEBUG oslo_concurrency.lockutils [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75557) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 862.219351] env[75557]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 862.219516] env[75557]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75557) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 864.220951] env[75557]: CRITICAL neutron [None req-e256e907-b8e6-4818-b00b-6c56be2cc526 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 864.220951] env[75557]: ERROR neutron Traceback (most recent call last): [ 864.220951] env[75557]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 864.220951] env[75557]: ERROR neutron sys.exit(main()) [ 864.220951] env[75557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 864.220951] env[75557]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 864.220951] env[75557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 864.220951] env[75557]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 864.220951] env[75557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 864.220951] env[75557]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 864.220951] env[75557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 864.220951] env[75557]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 864.220951] env[75557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 864.220951] env[75557]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 864.220951] env[75557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 864.220951] env[75557]: ERROR neutron response = self.get(path=path, params=params) [ 864.221641] env[75557]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 864.221641] env[75557]: ERROR neutron raise RuntimeError(msg, last_err) [ 864.221641] env[75557]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 864.221641] env[75557]: ERROR neutron [ 866.859045] env[75696]: INFO neutron.common.config [-] Logging enabled! [ 866.859328] env[75696]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 866.859616] env[75696]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75696) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 866.859842] env[75696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 866.859928] env[75696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 866.860824] env[75696]: WARNING oslo_policy.policy [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 866.861039] env[75696]: WARNING oslo_policy.policy [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 866.862272] env[75696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 866.876740] env[75696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Session token - acquiring [ 866.877550] env[75696]: DEBUG networking_nsxv3.common.locking [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75696) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 866.877757] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 866.877884] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 866.878247] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 866.887523] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 866.887849] env[75696]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 866.888056] env[75696]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75696) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 868.894196] env[75696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Session token - acquiring [ 868.894518] env[75696]: DEBUG networking_nsxv3.common.locking [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75696) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 868.894518] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 868.894603] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 868.894817] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 868.904244] env[75696]: DEBUG oslo_concurrency.lockutils [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 868.904485] env[75696]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 868.904627] env[75696]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75696) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 870.907604] env[75696]: CRITICAL neutron [None req-0d74fbd6-c021-47ea-9858-1ca59ab042da None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 870.907604] env[75696]: ERROR neutron Traceback (most recent call last): [ 870.907604] env[75696]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 870.907604] env[75696]: ERROR neutron sys.exit(main()) [ 870.907604] env[75696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 870.907604] env[75696]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 870.907604] env[75696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 870.907604] env[75696]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 870.907604] env[75696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 870.907604] env[75696]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 870.907604] env[75696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 870.907604] env[75696]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 870.907604] env[75696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 870.907604] env[75696]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 870.907604] env[75696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 870.907604] env[75696]: ERROR neutron response = self.get(path=path, params=params) [ 870.908567] env[75696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 870.908567] env[75696]: ERROR neutron raise RuntimeError(msg, last_err) [ 870.908567] env[75696]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 870.908567] env[75696]: ERROR neutron [ 873.349471] env[75725]: INFO neutron.common.config [-] Logging enabled! [ 873.349807] env[75725]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 873.349969] env[75725]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75725) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 873.350262] env[75725]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 873.350305] env[75725]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 873.351135] env[75725]: WARNING oslo_policy.policy [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 873.351349] env[75725]: WARNING oslo_policy.policy [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 873.352650] env[75725]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 873.366851] env[75725]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Session token - acquiring [ 873.367711] env[75725]: DEBUG networking_nsxv3.common.locking [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75725) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 873.367911] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 873.368050] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 873.368412] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 873.377264] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 873.377547] env[75725]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 873.377701] env[75725]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75725) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 875.383813] env[75725]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Session token - acquiring [ 875.384199] env[75725]: DEBUG networking_nsxv3.common.locking [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75725) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 875.384418] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 875.384542] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 875.384808] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 875.394173] env[75725]: DEBUG oslo_concurrency.lockutils [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75725) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 875.394407] env[75725]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 875.394544] env[75725]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75725) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 877.397490] env[75725]: CRITICAL neutron [None req-86a20d23-9178-4d66-be57-4559e694a561 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 877.397490] env[75725]: ERROR neutron Traceback (most recent call last): [ 877.397490] env[75725]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 877.397490] env[75725]: ERROR neutron sys.exit(main()) [ 877.397490] env[75725]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 877.397490] env[75725]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 877.397490] env[75725]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 877.397490] env[75725]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 877.397490] env[75725]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 877.397490] env[75725]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 877.397490] env[75725]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 877.397490] env[75725]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 877.397490] env[75725]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 877.397490] env[75725]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 877.397490] env[75725]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 877.397490] env[75725]: ERROR neutron response = self.get(path=path, params=params) [ 877.398192] env[75725]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 877.398192] env[75725]: ERROR neutron raise RuntimeError(msg, last_err) [ 877.398192] env[75725]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 877.398192] env[75725]: ERROR neutron [ 879.900025] env[75752]: INFO neutron.common.config [-] Logging enabled! [ 879.900025] env[75752]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 879.900025] env[75752]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75752) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 879.900025] env[75752]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 879.900025] env[75752]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 879.900025] env[75752]: WARNING oslo_policy.policy [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 879.900025] env[75752]: WARNING oslo_policy.policy [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 879.900577] env[75752]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 879.921254] env[75752]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Session token - acquiring [ 879.922115] env[75752]: DEBUG networking_nsxv3.common.locking [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75752) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 879.922327] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 879.922455] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 879.922836] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 879.934499] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 879.934749] env[75752]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 879.934905] env[75752]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75752) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 881.940948] env[75752]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Session token - acquiring [ 881.941378] env[75752]: DEBUG networking_nsxv3.common.locking [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75752) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 881.941452] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 881.941588] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 881.941869] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 881.951229] env[75752]: DEBUG oslo_concurrency.lockutils [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75752) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 881.951469] env[75752]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 881.951613] env[75752]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75752) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 883.952889] env[75752]: CRITICAL neutron [None req-598b90c8-633e-4efd-a560-a98183b4259a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 883.952889] env[75752]: ERROR neutron Traceback (most recent call last): [ 883.952889] env[75752]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 883.952889] env[75752]: ERROR neutron sys.exit(main()) [ 883.952889] env[75752]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 883.952889] env[75752]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 883.952889] env[75752]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 883.952889] env[75752]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 883.952889] env[75752]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 883.952889] env[75752]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 883.952889] env[75752]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 883.952889] env[75752]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 883.952889] env[75752]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 883.952889] env[75752]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 883.952889] env[75752]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 883.952889] env[75752]: ERROR neutron response = self.get(path=path, params=params) [ 883.953965] env[75752]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 883.953965] env[75752]: ERROR neutron raise RuntimeError(msg, last_err) [ 883.953965] env[75752]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 883.953965] env[75752]: ERROR neutron [ 886.580193] env[75849]: INFO neutron.common.config [-] Logging enabled! [ 886.580556] env[75849]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 886.580749] env[75849]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75849) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 886.580972] env[75849]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 886.581065] env[75849]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 886.581879] env[75849]: WARNING oslo_policy.policy [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 886.582098] env[75849]: WARNING oslo_policy.policy [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 886.583284] env[75849]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 886.597846] env[75849]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Session token - acquiring [ 886.598470] env[75849]: DEBUG networking_nsxv3.common.locking [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75849) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 886.598674] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 886.598858] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 886.599219] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 886.608567] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 886.608815] env[75849]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 886.608966] env[75849]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75849) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 888.614946] env[75849]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Session token - acquiring [ 888.615333] env[75849]: DEBUG networking_nsxv3.common.locking [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75849) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 888.615530] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 888.615657] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 888.615930] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 888.624977] env[75849]: DEBUG oslo_concurrency.lockutils [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75849) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 888.625275] env[75849]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 888.625451] env[75849]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75849) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 890.628601] env[75849]: CRITICAL neutron [None req-78491ab7-91a1-4995-a53e-79a87964a828 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 890.628601] env[75849]: ERROR neutron Traceback (most recent call last): [ 890.628601] env[75849]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 890.628601] env[75849]: ERROR neutron sys.exit(main()) [ 890.628601] env[75849]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 890.628601] env[75849]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 890.628601] env[75849]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 890.628601] env[75849]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 890.628601] env[75849]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 890.628601] env[75849]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 890.628601] env[75849]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 890.628601] env[75849]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 890.628601] env[75849]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 890.628601] env[75849]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 890.628601] env[75849]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 890.628601] env[75849]: ERROR neutron response = self.get(path=path, params=params) [ 890.629246] env[75849]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 890.629246] env[75849]: ERROR neutron raise RuntimeError(msg, last_err) [ 890.629246] env[75849]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 890.629246] env[75849]: ERROR neutron [ 893.064232] env[75895]: INFO neutron.common.config [-] Logging enabled! [ 893.064530] env[75895]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 893.064765] env[75895]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75895) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 893.065009] env[75895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 893.065105] env[75895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 893.065939] env[75895]: WARNING oslo_policy.policy [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 893.066175] env[75895]: WARNING oslo_policy.policy [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 893.067381] env[75895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 893.081495] env[75895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Session token - acquiring [ 893.082315] env[75895]: DEBUG networking_nsxv3.common.locking [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75895) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 893.082520] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 893.082652] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 893.083022] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 893.092480] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 893.092723] env[75895]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 893.092880] env[75895]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75895) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 895.098825] env[75895]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Session token - acquiring [ 895.099265] env[75895]: DEBUG networking_nsxv3.common.locking [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75895) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 895.099265] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 895.099444] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 895.099695] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 895.108904] env[75895]: DEBUG oslo_concurrency.lockutils [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75895) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 895.109108] env[75895]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 895.109260] env[75895]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75895) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 897.112268] env[75895]: CRITICAL neutron [None req-d48be4d6-59e6-4df1-84cb-d89a69996f2d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 897.112268] env[75895]: ERROR neutron Traceback (most recent call last): [ 897.112268] env[75895]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 897.112268] env[75895]: ERROR neutron sys.exit(main()) [ 897.112268] env[75895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 897.112268] env[75895]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 897.112268] env[75895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 897.112268] env[75895]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 897.112268] env[75895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 897.112268] env[75895]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 897.112268] env[75895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 897.112268] env[75895]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 897.112268] env[75895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 897.112268] env[75895]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 897.112268] env[75895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 897.112268] env[75895]: ERROR neutron response = self.get(path=path, params=params) [ 897.113055] env[75895]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 897.113055] env[75895]: ERROR neutron raise RuntimeError(msg, last_err) [ 897.113055] env[75895]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 897.113055] env[75895]: ERROR neutron [ 899.634609] env[75913]: INFO neutron.common.config [-] Logging enabled! [ 899.634982] env[75913]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 899.635155] env[75913]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75913) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 899.635391] env[75913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 899.635485] env[75913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 899.636373] env[75913]: WARNING oslo_policy.policy [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 899.636597] env[75913]: WARNING oslo_policy.policy [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 899.637797] env[75913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 899.652921] env[75913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Session token - acquiring [ 899.654225] env[75913]: DEBUG networking_nsxv3.common.locking [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75913) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 899.654439] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 899.654569] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 899.654929] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 899.666035] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 899.666279] env[75913]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 899.666435] env[75913]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75913) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 901.672350] env[75913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Session token - acquiring [ 901.672704] env[75913]: DEBUG networking_nsxv3.common.locking [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75913) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 901.672833] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 901.672942] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 901.673221] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 901.681968] env[75913]: DEBUG oslo_concurrency.lockutils [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 901.682194] env[75913]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 901.682329] env[75913]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75913) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 903.684881] env[75913]: CRITICAL neutron [None req-e4b0878d-49f8-42eb-bc49-a91679570de3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 903.684881] env[75913]: ERROR neutron Traceback (most recent call last): [ 903.684881] env[75913]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 903.684881] env[75913]: ERROR neutron sys.exit(main()) [ 903.684881] env[75913]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 903.684881] env[75913]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 903.684881] env[75913]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 903.684881] env[75913]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 903.684881] env[75913]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 903.684881] env[75913]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 903.684881] env[75913]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 903.684881] env[75913]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 903.684881] env[75913]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 903.684881] env[75913]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 903.684881] env[75913]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 903.684881] env[75913]: ERROR neutron response = self.get(path=path, params=params) [ 903.685537] env[75913]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 903.685537] env[75913]: ERROR neutron raise RuntimeError(msg, last_err) [ 903.685537] env[75913]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 903.685537] env[75913]: ERROR neutron [ 906.478446] env[76026]: INFO neutron.common.config [-] Logging enabled! [ 906.478680] env[76026]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 906.479093] env[76026]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76026) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 906.479352] env[76026]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 906.479462] env[76026]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 906.480303] env[76026]: WARNING oslo_policy.policy [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 906.480516] env[76026]: WARNING oslo_policy.policy [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 906.481753] env[76026]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 906.496585] env[76026]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Session token - acquiring [ 906.497182] env[76026]: DEBUG networking_nsxv3.common.locking [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76026) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 906.497431] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 906.497587] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 906.497973] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 906.515620] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 906.515620] env[76026]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 906.515620] env[76026]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76026) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 908.520422] env[76026]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Session token - acquiring [ 908.520815] env[76026]: DEBUG networking_nsxv3.common.locking [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76026) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 908.521005] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 908.521124] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 908.521394] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 908.530172] env[76026]: DEBUG oslo_concurrency.lockutils [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76026) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 908.530403] env[76026]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 908.530540] env[76026]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76026) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 910.532918] env[76026]: CRITICAL neutron [None req-ff8ce267-c6ed-4bbb-9b05-77ec07768fd6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 910.532918] env[76026]: ERROR neutron Traceback (most recent call last): [ 910.532918] env[76026]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 910.532918] env[76026]: ERROR neutron sys.exit(main()) [ 910.532918] env[76026]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 910.532918] env[76026]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 910.532918] env[76026]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 910.532918] env[76026]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 910.532918] env[76026]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 910.532918] env[76026]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 910.532918] env[76026]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 910.532918] env[76026]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 910.532918] env[76026]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 910.532918] env[76026]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 910.532918] env[76026]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 910.532918] env[76026]: ERROR neutron response = self.get(path=path, params=params) [ 910.533631] env[76026]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 910.533631] env[76026]: ERROR neutron raise RuntimeError(msg, last_err) [ 910.533631] env[76026]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 910.533631] env[76026]: ERROR neutron [ 913.257390] env[76243]: INFO neutron.common.config [-] Logging enabled! [ 913.257390] env[76243]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 913.257390] env[76243]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76243) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 913.257390] env[76243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 913.257390] env[76243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 913.257390] env[76243]: WARNING oslo_policy.policy [None req-2535e946-df6b-4f86-84de-81a60453803d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 913.257390] env[76243]: WARNING oslo_policy.policy [None req-2535e946-df6b-4f86-84de-81a60453803d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 913.257390] env[76243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 913.271298] env[76243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Session token - acquiring [ 913.272152] env[76243]: DEBUG networking_nsxv3.common.locking [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76243) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 913.272356] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 913.272489] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 913.272870] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 913.282303] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 913.282548] env[76243]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 913.282692] env[76243]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76243) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 915.291011] env[76243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Session token - acquiring [ 915.291011] env[76243]: DEBUG networking_nsxv3.common.locking [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76243) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 915.291011] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 915.291481] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 915.291481] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 915.299270] env[76243]: DEBUG oslo_concurrency.lockutils [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 915.299530] env[76243]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 915.299671] env[76243]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76243) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 917.300894] env[76243]: CRITICAL neutron [None req-2535e946-df6b-4f86-84de-81a60453803d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 917.300894] env[76243]: ERROR neutron Traceback (most recent call last): [ 917.300894] env[76243]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 917.300894] env[76243]: ERROR neutron sys.exit(main()) [ 917.300894] env[76243]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 917.300894] env[76243]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 917.300894] env[76243]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 917.300894] env[76243]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 917.300894] env[76243]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 917.300894] env[76243]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 917.300894] env[76243]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 917.300894] env[76243]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 917.300894] env[76243]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 917.300894] env[76243]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 917.300894] env[76243]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 917.300894] env[76243]: ERROR neutron response = self.get(path=path, params=params) [ 917.301603] env[76243]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 917.301603] env[76243]: ERROR neutron raise RuntimeError(msg, last_err) [ 917.301603] env[76243]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 917.301603] env[76243]: ERROR neutron [ 919.979286] env[76565]: INFO neutron.common.config [-] Logging enabled! [ 919.979606] env[76565]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 919.979845] env[76565]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76565) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 919.980092] env[76565]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 919.980195] env[76565]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 919.981911] env[76565]: WARNING oslo_policy.policy [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 919.982143] env[76565]: WARNING oslo_policy.policy [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 919.983466] env[76565]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 919.997933] env[76565]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Session token - acquiring [ 919.998806] env[76565]: DEBUG networking_nsxv3.common.locking [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76565) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 919.999029] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 919.999159] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 919.999552] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 920.017564] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 920.017935] env[76565]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 920.018008] env[76565]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76565) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 922.025887] env[76565]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Session token - acquiring [ 922.026213] env[76565]: DEBUG networking_nsxv3.common.locking [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76565) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 922.026419] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 922.026547] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 922.026994] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 922.035869] env[76565]: DEBUG oslo_concurrency.lockutils [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76565) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 922.036166] env[76565]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 922.036264] env[76565]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76565) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 924.039818] env[76565]: CRITICAL neutron [None req-1f3aa56d-ad86-4b27-b1c9-fbc7df6dab0f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 924.039818] env[76565]: ERROR neutron Traceback (most recent call last): [ 924.039818] env[76565]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 924.039818] env[76565]: ERROR neutron sys.exit(main()) [ 924.039818] env[76565]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 924.039818] env[76565]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 924.039818] env[76565]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 924.039818] env[76565]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 924.039818] env[76565]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 924.039818] env[76565]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 924.039818] env[76565]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 924.039818] env[76565]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 924.039818] env[76565]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 924.039818] env[76565]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 924.039818] env[76565]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 924.039818] env[76565]: ERROR neutron response = self.get(path=path, params=params) [ 924.040467] env[76565]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 924.040467] env[76565]: ERROR neutron raise RuntimeError(msg, last_err) [ 924.040467] env[76565]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 924.040467] env[76565]: ERROR neutron [ 926.804608] env[76814]: INFO neutron.common.config [-] Logging enabled! [ 926.804958] env[76814]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 926.805130] env[76814]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76814) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 926.805354] env[76814]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 926.805445] env[76814]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 926.806279] env[76814]: WARNING oslo_policy.policy [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 926.806507] env[76814]: WARNING oslo_policy.policy [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 926.807782] env[76814]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 926.821951] env[76814]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Session token - acquiring [ 926.822772] env[76814]: DEBUG networking_nsxv3.common.locking [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76814) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 926.823051] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 926.823152] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 926.823577] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 926.833101] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 926.833383] env[76814]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 926.833550] env[76814]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76814) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 928.839768] env[76814]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Session token - acquiring [ 928.840117] env[76814]: DEBUG networking_nsxv3.common.locking [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76814) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 928.840336] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 928.840479] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 928.840891] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 928.850459] env[76814]: DEBUG oslo_concurrency.lockutils [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76814) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 928.850769] env[76814]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 928.850950] env[76814]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76814) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 930.852979] env[76814]: CRITICAL neutron [None req-227d7bb9-0a4a-45bc-82e3-a0f9cb60df74 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 930.852979] env[76814]: ERROR neutron Traceback (most recent call last): [ 930.852979] env[76814]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 930.852979] env[76814]: ERROR neutron sys.exit(main()) [ 930.852979] env[76814]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 930.852979] env[76814]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 930.852979] env[76814]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 930.852979] env[76814]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 930.852979] env[76814]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 930.852979] env[76814]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 930.852979] env[76814]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 930.852979] env[76814]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 930.852979] env[76814]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 930.852979] env[76814]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 930.852979] env[76814]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 930.852979] env[76814]: ERROR neutron response = self.get(path=path, params=params) [ 930.853704] env[76814]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 930.853704] env[76814]: ERROR neutron raise RuntimeError(msg, last_err) [ 930.853704] env[76814]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 930.853704] env[76814]: ERROR neutron [ 933.374507] env[76921]: INFO neutron.common.config [-] Logging enabled! [ 933.374800] env[76921]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 933.374983] env[76921]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76921) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 933.375233] env[76921]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 933.375327] env[76921]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 933.376189] env[76921]: WARNING oslo_policy.policy [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 933.376419] env[76921]: WARNING oslo_policy.policy [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 933.377668] env[76921]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 933.393785] env[76921]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Session token - acquiring [ 933.394627] env[76921]: DEBUG networking_nsxv3.common.locking [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76921) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 933.394835] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 933.394962] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 933.395336] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 933.404631] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 933.404900] env[76921]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 933.405134] env[76921]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76921) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 935.411296] env[76921]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Session token - acquiring [ 935.411711] env[76921]: DEBUG networking_nsxv3.common.locking [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76921) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 935.411908] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 935.412050] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 935.412335] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 935.421097] env[76921]: DEBUG oslo_concurrency.lockutils [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76921) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 935.421340] env[76921]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 935.421481] env[76921]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76921) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 937.424217] env[76921]: CRITICAL neutron [None req-b8541128-64f8-4047-b64f-af3aceb98751 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 937.424217] env[76921]: ERROR neutron Traceback (most recent call last): [ 937.424217] env[76921]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 937.424217] env[76921]: ERROR neutron sys.exit(main()) [ 937.424217] env[76921]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 937.424217] env[76921]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 937.424217] env[76921]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 937.424217] env[76921]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 937.424217] env[76921]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 937.424217] env[76921]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 937.424217] env[76921]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 937.424217] env[76921]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 937.424217] env[76921]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 937.424217] env[76921]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 937.424217] env[76921]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 937.424217] env[76921]: ERROR neutron response = self.get(path=path, params=params) [ 937.424871] env[76921]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 937.424871] env[76921]: ERROR neutron raise RuntimeError(msg, last_err) [ 937.424871] env[76921]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 937.424871] env[76921]: ERROR neutron [ 940.086544] env[76959]: INFO neutron.common.config [-] Logging enabled! [ 940.086868] env[76959]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 940.087078] env[76959]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76959) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 940.087323] env[76959]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 940.087415] env[76959]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 940.088313] env[76959]: WARNING oslo_policy.policy [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 940.088534] env[76959]: WARNING oslo_policy.policy [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 940.089777] env[76959]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 940.104056] env[76959]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Session token - acquiring [ 940.104856] env[76959]: DEBUG networking_nsxv3.common.locking [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76959) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 940.105057] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 940.105182] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 940.105543] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 940.115351] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 940.115607] env[76959]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 940.115758] env[76959]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76959) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 942.121881] env[76959]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Session token - acquiring [ 942.122223] env[76959]: DEBUG networking_nsxv3.common.locking [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76959) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 942.122344] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 942.122462] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 942.122747] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 942.132252] env[76959]: DEBUG oslo_concurrency.lockutils [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76959) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 942.132568] env[76959]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 942.132685] env[76959]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76959) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 944.135716] env[76959]: CRITICAL neutron [None req-5800990e-6c7c-4f5b-82b8-7f2758f872cb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 944.135716] env[76959]: ERROR neutron Traceback (most recent call last): [ 944.135716] env[76959]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 944.135716] env[76959]: ERROR neutron sys.exit(main()) [ 944.135716] env[76959]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 944.135716] env[76959]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 944.135716] env[76959]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 944.135716] env[76959]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 944.135716] env[76959]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 944.135716] env[76959]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 944.135716] env[76959]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 944.135716] env[76959]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 944.135716] env[76959]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 944.135716] env[76959]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 944.135716] env[76959]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 944.135716] env[76959]: ERROR neutron response = self.get(path=path, params=params) [ 944.136510] env[76959]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 944.136510] env[76959]: ERROR neutron raise RuntimeError(msg, last_err) [ 944.136510] env[76959]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 944.136510] env[76959]: ERROR neutron [ 946.576890] env[77051]: INFO neutron.common.config [-] Logging enabled! [ 946.577213] env[77051]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 946.577451] env[77051]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77051) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 946.577674] env[77051]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 946.577770] env[77051]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 946.578651] env[77051]: WARNING oslo_policy.policy [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 946.578849] env[77051]: WARNING oslo_policy.policy [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 946.580083] env[77051]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 946.594194] env[77051]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Session token - acquiring [ 946.594971] env[77051]: DEBUG networking_nsxv3.common.locking [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77051) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 946.595171] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 946.595342] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 946.595645] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 946.604658] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 946.604908] env[77051]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 946.605056] env[77051]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77051) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 948.611047] env[77051]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Session token - acquiring [ 948.611465] env[77051]: DEBUG networking_nsxv3.common.locking [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77051) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 948.611552] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 948.611673] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 948.611962] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 948.621059] env[77051]: DEBUG oslo_concurrency.lockutils [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77051) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 948.621293] env[77051]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 948.621433] env[77051]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77051) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 950.623571] env[77051]: CRITICAL neutron [None req-d5034b23-d31e-4b4a-a565-87f5b93828b9 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 950.623571] env[77051]: ERROR neutron Traceback (most recent call last): [ 950.623571] env[77051]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 950.623571] env[77051]: ERROR neutron sys.exit(main()) [ 950.623571] env[77051]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 950.623571] env[77051]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 950.623571] env[77051]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 950.623571] env[77051]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 950.623571] env[77051]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 950.623571] env[77051]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 950.623571] env[77051]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 950.623571] env[77051]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 950.623571] env[77051]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 950.623571] env[77051]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 950.623571] env[77051]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 950.623571] env[77051]: ERROR neutron response = self.get(path=path, params=params) [ 950.624736] env[77051]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 950.624736] env[77051]: ERROR neutron raise RuntimeError(msg, last_err) [ 950.624736] env[77051]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 950.624736] env[77051]: ERROR neutron [ 953.145754] env[77141]: INFO neutron.common.config [-] Logging enabled! [ 953.146052] env[77141]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 953.146277] env[77141]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77141) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 953.146505] env[77141]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 953.146602] env[77141]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 953.147439] env[77141]: WARNING oslo_policy.policy [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 953.147664] env[77141]: WARNING oslo_policy.policy [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 953.148922] env[77141]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 953.163114] env[77141]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Session token - acquiring [ 953.164290] env[77141]: DEBUG networking_nsxv3.common.locking [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77141) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 953.164523] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 953.164654] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 953.165026] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 953.173878] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 953.174126] env[77141]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 953.174276] env[77141]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77141) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 955.180301] env[77141]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Session token - acquiring [ 955.180704] env[77141]: DEBUG networking_nsxv3.common.locking [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77141) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 955.181990] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 955.181990] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 955.181990] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 955.190839] env[77141]: DEBUG oslo_concurrency.lockutils [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77141) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 955.191081] env[77141]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 955.191215] env[77141]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77141) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 957.192915] env[77141]: CRITICAL neutron [None req-e71d15d8-c219-4ec2-8d0c-8c9eeaa420f8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 957.192915] env[77141]: ERROR neutron Traceback (most recent call last): [ 957.192915] env[77141]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 957.192915] env[77141]: ERROR neutron sys.exit(main()) [ 957.192915] env[77141]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 957.192915] env[77141]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 957.192915] env[77141]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 957.192915] env[77141]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 957.192915] env[77141]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 957.192915] env[77141]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 957.192915] env[77141]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 957.192915] env[77141]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 957.192915] env[77141]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 957.192915] env[77141]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 957.192915] env[77141]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 957.192915] env[77141]: ERROR neutron response = self.get(path=path, params=params) [ 957.193753] env[77141]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 957.193753] env[77141]: ERROR neutron raise RuntimeError(msg, last_err) [ 957.193753] env[77141]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 957.193753] env[77141]: ERROR neutron [ 959.815689] env[77207]: INFO neutron.common.config [-] Logging enabled! [ 959.816116] env[77207]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 959.816233] env[77207]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77207) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 959.816450] env[77207]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 959.816544] env[77207]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 959.817378] env[77207]: WARNING oslo_policy.policy [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 959.817589] env[77207]: WARNING oslo_policy.policy [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 959.818782] env[77207]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 959.833137] env[77207]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Session token - acquiring [ 959.833889] env[77207]: DEBUG networking_nsxv3.common.locking [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77207) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 959.834086] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 959.834212] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 959.834561] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 959.843687] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 959.843933] env[77207]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 959.844104] env[77207]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77207) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 961.850531] env[77207]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Session token - acquiring [ 961.850945] env[77207]: DEBUG networking_nsxv3.common.locking [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77207) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 961.851195] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 961.851387] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 961.851736] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 961.861471] env[77207]: DEBUG oslo_concurrency.lockutils [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77207) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 961.861834] env[77207]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 961.862086] env[77207]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77207) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 963.864932] env[77207]: CRITICAL neutron [None req-89892e3a-bd2d-486a-abcb-1e6d25bc9a59 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 963.864932] env[77207]: ERROR neutron Traceback (most recent call last): [ 963.864932] env[77207]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 963.864932] env[77207]: ERROR neutron sys.exit(main()) [ 963.864932] env[77207]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 963.864932] env[77207]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 963.864932] env[77207]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 963.864932] env[77207]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 963.864932] env[77207]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 963.864932] env[77207]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 963.864932] env[77207]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 963.864932] env[77207]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 963.864932] env[77207]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 963.864932] env[77207]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 963.864932] env[77207]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 963.864932] env[77207]: ERROR neutron response = self.get(path=path, params=params) [ 963.865702] env[77207]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 963.865702] env[77207]: ERROR neutron raise RuntimeError(msg, last_err) [ 963.865702] env[77207]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 963.865702] env[77207]: ERROR neutron [ 966.650896] env[77376]: INFO neutron.common.config [-] Logging enabled! [ 966.651260] env[77376]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 966.651360] env[77376]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77376) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 966.651590] env[77376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 966.651685] env[77376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 966.652522] env[77376]: WARNING oslo_policy.policy [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 966.652732] env[77376]: WARNING oslo_policy.policy [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 966.653866] env[77376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 966.667915] env[77376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Session token - acquiring [ 966.668631] env[77376]: DEBUG networking_nsxv3.common.locking [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77376) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 966.668830] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 966.668955] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 966.669305] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 966.677927] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 966.678167] env[77376]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 966.678314] env[77376]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77376) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 968.684313] env[77376]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Session token - acquiring [ 968.684647] env[77376]: DEBUG networking_nsxv3.common.locking [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77376) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 968.684834] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 968.684956] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 968.685228] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 968.694279] env[77376]: DEBUG oslo_concurrency.lockutils [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77376) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 968.694505] env[77376]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 968.694643] env[77376]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77376) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 970.697667] env[77376]: CRITICAL neutron [None req-78531162-f942-4a41-abdb-872cc2b8d434 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 970.697667] env[77376]: ERROR neutron Traceback (most recent call last): [ 970.697667] env[77376]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 970.697667] env[77376]: ERROR neutron sys.exit(main()) [ 970.697667] env[77376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 970.697667] env[77376]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 970.697667] env[77376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 970.697667] env[77376]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 970.697667] env[77376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 970.697667] env[77376]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 970.697667] env[77376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 970.697667] env[77376]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 970.697667] env[77376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 970.697667] env[77376]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 970.697667] env[77376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 970.697667] env[77376]: ERROR neutron response = self.get(path=path, params=params) [ 970.698562] env[77376]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 970.698562] env[77376]: ERROR neutron raise RuntimeError(msg, last_err) [ 970.698562] env[77376]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 970.698562] env[77376]: ERROR neutron [ 973.110505] env[77529]: INFO neutron.common.config [-] Logging enabled! [ 973.110944] env[77529]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 973.111052] env[77529]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77529) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 973.111338] env[77529]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 973.111432] env[77529]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 973.112294] env[77529]: WARNING oslo_policy.policy [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 973.112520] env[77529]: WARNING oslo_policy.policy [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 973.113728] env[77529]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 973.127853] env[77529]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Session token - acquiring [ 973.128629] env[77529]: DEBUG networking_nsxv3.common.locking [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77529) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 973.128835] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 973.128962] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 973.129317] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 973.138509] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 973.138765] env[77529]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 973.138913] env[77529]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77529) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 975.146955] env[77529]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Session token - acquiring [ 975.147450] env[77529]: DEBUG networking_nsxv3.common.locking [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77529) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 975.147539] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 975.147595] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 975.147865] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 975.157171] env[77529]: DEBUG oslo_concurrency.lockutils [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77529) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 975.157412] env[77529]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 975.157543] env[77529]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77529) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 977.160459] env[77529]: CRITICAL neutron [None req-7d510236-3260-4947-bbc4-6b2b96542305 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 977.160459] env[77529]: ERROR neutron Traceback (most recent call last): [ 977.160459] env[77529]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 977.160459] env[77529]: ERROR neutron sys.exit(main()) [ 977.160459] env[77529]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 977.160459] env[77529]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 977.160459] env[77529]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 977.160459] env[77529]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 977.160459] env[77529]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 977.160459] env[77529]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 977.160459] env[77529]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 977.160459] env[77529]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 977.160459] env[77529]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 977.160459] env[77529]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 977.160459] env[77529]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 977.160459] env[77529]: ERROR neutron response = self.get(path=path, params=params) [ 977.161273] env[77529]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 977.161273] env[77529]: ERROR neutron raise RuntimeError(msg, last_err) [ 977.161273] env[77529]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 977.161273] env[77529]: ERROR neutron [ 979.614997] env[77603]: INFO neutron.common.config [-] Logging enabled! [ 979.615275] env[77603]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 979.615530] env[77603]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77603) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 979.615774] env[77603]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 979.615868] env[77603]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 979.616729] env[77603]: WARNING oslo_policy.policy [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 979.616959] env[77603]: WARNING oslo_policy.policy [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 979.618182] env[77603]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 979.633488] env[77603]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Session token - acquiring [ 979.633488] env[77603]: DEBUG networking_nsxv3.common.locking [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77603) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 979.633673] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 979.633777] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 979.634136] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 979.643674] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 979.643922] env[77603]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 979.644087] env[77603]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77603) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 981.650274] env[77603]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Session token - acquiring [ 981.650638] env[77603]: DEBUG networking_nsxv3.common.locking [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77603) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 981.650864] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 981.651000] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 981.651317] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 981.660696] env[77603]: DEBUG oslo_concurrency.lockutils [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77603) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 981.660940] env[77603]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 981.661082] env[77603]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77603) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 983.663996] env[77603]: CRITICAL neutron [None req-e2bcbe14-b296-415f-9af6-b43a44c6b4cb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 983.663996] env[77603]: ERROR neutron Traceback (most recent call last): [ 983.663996] env[77603]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 983.663996] env[77603]: ERROR neutron sys.exit(main()) [ 983.663996] env[77603]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 983.663996] env[77603]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 983.663996] env[77603]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 983.663996] env[77603]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 983.663996] env[77603]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 983.663996] env[77603]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 983.663996] env[77603]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 983.663996] env[77603]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 983.663996] env[77603]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 983.663996] env[77603]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 983.663996] env[77603]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 983.663996] env[77603]: ERROR neutron response = self.get(path=path, params=params) [ 983.664768] env[77603]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 983.664768] env[77603]: ERROR neutron raise RuntimeError(msg, last_err) [ 983.664768] env[77603]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 983.664768] env[77603]: ERROR neutron [ 986.253051] env[77696]: INFO neutron.common.config [-] Logging enabled! [ 986.253437] env[77696]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 986.253557] env[77696]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77696) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 986.253784] env[77696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 986.253880] env[77696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 986.254711] env[77696]: WARNING oslo_policy.policy [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 986.254929] env[77696]: WARNING oslo_policy.policy [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 986.256098] env[77696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 986.269877] env[77696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Session token - acquiring [ 986.270630] env[77696]: DEBUG networking_nsxv3.common.locking [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77696) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 986.270830] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 986.270957] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 986.271306] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 986.280293] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 986.280566] env[77696]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 986.280724] env[77696]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77696) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 988.288912] env[77696]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Session token - acquiring [ 988.289387] env[77696]: DEBUG networking_nsxv3.common.locking [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77696) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 988.289627] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 988.289757] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 988.290037] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 988.299351] env[77696]: DEBUG oslo_concurrency.lockutils [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77696) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 988.299605] env[77696]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 988.299746] env[77696]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77696) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 990.301150] env[77696]: CRITICAL neutron [None req-8b29cfa4-ebe8-4b98-8a97-30b03c04f76c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 990.301150] env[77696]: ERROR neutron Traceback (most recent call last): [ 990.301150] env[77696]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 990.301150] env[77696]: ERROR neutron sys.exit(main()) [ 990.301150] env[77696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 990.301150] env[77696]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 990.301150] env[77696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 990.301150] env[77696]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 990.301150] env[77696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 990.301150] env[77696]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 990.301150] env[77696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 990.301150] env[77696]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 990.301150] env[77696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 990.301150] env[77696]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 990.301150] env[77696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 990.301150] env[77696]: ERROR neutron response = self.get(path=path, params=params) [ 990.301985] env[77696]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 990.301985] env[77696]: ERROR neutron raise RuntimeError(msg, last_err) [ 990.301985] env[77696]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 990.301985] env[77696]: ERROR neutron [ 992.807594] env[77749]: INFO neutron.common.config [-] Logging enabled! [ 992.807930] env[77749]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 992.808129] env[77749]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77749) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 992.808353] env[77749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 992.808448] env[77749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 992.809287] env[77749]: WARNING oslo_policy.policy [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 992.809515] env[77749]: WARNING oslo_policy.policy [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 992.810714] env[77749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 992.824820] env[77749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Session token - acquiring [ 992.825645] env[77749]: DEBUG networking_nsxv3.common.locking [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77749) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 992.825862] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 992.826100] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 992.826721] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 992.835919] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 992.836205] env[77749]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 992.836359] env[77749]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77749) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 994.845687] env[77749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Session token - acquiring [ 994.846139] env[77749]: DEBUG networking_nsxv3.common.locking [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77749) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 994.846388] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 994.846471] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 994.846727] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 994.856736] env[77749]: DEBUG oslo_concurrency.lockutils [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 994.857041] env[77749]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 994.857209] env[77749]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77749) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 996.858363] env[77749]: CRITICAL neutron [None req-c2583bff-9eed-4e55-b812-da5b87af1f10 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 996.858363] env[77749]: ERROR neutron Traceback (most recent call last): [ 996.858363] env[77749]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 996.858363] env[77749]: ERROR neutron sys.exit(main()) [ 996.858363] env[77749]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 996.858363] env[77749]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 996.858363] env[77749]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 996.858363] env[77749]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 996.858363] env[77749]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 996.858363] env[77749]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 996.858363] env[77749]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 996.858363] env[77749]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 996.858363] env[77749]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 996.858363] env[77749]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 996.858363] env[77749]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 996.858363] env[77749]: ERROR neutron response = self.get(path=path, params=params) [ 996.859151] env[77749]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 996.859151] env[77749]: ERROR neutron raise RuntimeError(msg, last_err) [ 996.859151] env[77749]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 996.859151] env[77749]: ERROR neutron [ 999.522610] env[77754]: INFO neutron.common.config [-] Logging enabled! [ 999.522863] env[77754]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 999.523185] env[77754]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77754) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 999.523393] env[77754]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 999.523489] env[77754]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 999.524344] env[77754]: WARNING oslo_policy.policy [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 999.524563] env[77754]: WARNING oslo_policy.policy [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 999.525784] env[77754]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 999.540574] env[77754]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Session token - acquiring [ 999.541443] env[77754]: DEBUG networking_nsxv3.common.locking [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77754) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 999.541671] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 999.541886] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 999.542282] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 999.551834] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 999.552222] env[77754]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 999.552413] env[77754]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77754) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1001.558625] env[77754]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Session token - acquiring [ 1001.559018] env[77754]: DEBUG networking_nsxv3.common.locking [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77754) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1001.559156] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1001.559280] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1001.559719] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1001.569096] env[77754]: DEBUG oslo_concurrency.lockutils [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77754) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1001.569351] env[77754]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1001.572155] env[77754]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77754) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1003.574492] env[77754]: CRITICAL neutron [None req-796510e0-a5a2-4dbc-a7d4-20f002050bb5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1003.574492] env[77754]: ERROR neutron Traceback (most recent call last): [ 1003.574492] env[77754]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1003.574492] env[77754]: ERROR neutron sys.exit(main()) [ 1003.574492] env[77754]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1003.574492] env[77754]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1003.574492] env[77754]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1003.574492] env[77754]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1003.574492] env[77754]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1003.574492] env[77754]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1003.574492] env[77754]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1003.574492] env[77754]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1003.574492] env[77754]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1003.574492] env[77754]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1003.574492] env[77754]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1003.574492] env[77754]: ERROR neutron response = self.get(path=path, params=params) [ 1003.575339] env[77754]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1003.575339] env[77754]: ERROR neutron raise RuntimeError(msg, last_err) [ 1003.575339] env[77754]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1003.575339] env[77754]: ERROR neutron [ 1005.993473] env[77825]: INFO neutron.common.config [-] Logging enabled! [ 1005.993892] env[77825]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1005.993982] env[77825]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77825) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1005.994218] env[77825]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1005.994310] env[77825]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1005.995134] env[77825]: WARNING oslo_policy.policy [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1005.995358] env[77825]: WARNING oslo_policy.policy [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1005.996601] env[77825]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1006.010489] env[77825]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Session token - acquiring [ 1006.012218] env[77825]: DEBUG networking_nsxv3.common.locking [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77825) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1006.012405] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1006.012537] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1006.012889] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1006.028299] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1006.028550] env[77825]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1006.028702] env[77825]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77825) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1008.035657] env[77825]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Session token - acquiring [ 1008.036036] env[77825]: DEBUG networking_nsxv3.common.locking [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77825) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1008.036186] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1008.036361] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1008.036586] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1008.045661] env[77825]: DEBUG oslo_concurrency.lockutils [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77825) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1008.045903] env[77825]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1008.046037] env[77825]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77825) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1010.048883] env[77825]: CRITICAL neutron [None req-7837985e-5574-4272-9eba-a9ba5d0015a1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1010.048883] env[77825]: ERROR neutron Traceback (most recent call last): [ 1010.048883] env[77825]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1010.048883] env[77825]: ERROR neutron sys.exit(main()) [ 1010.048883] env[77825]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1010.048883] env[77825]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1010.048883] env[77825]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1010.048883] env[77825]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1010.048883] env[77825]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1010.048883] env[77825]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1010.048883] env[77825]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1010.048883] env[77825]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1010.048883] env[77825]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1010.048883] env[77825]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1010.048883] env[77825]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1010.048883] env[77825]: ERROR neutron response = self.get(path=path, params=params) [ 1010.049632] env[77825]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1010.049632] env[77825]: ERROR neutron raise RuntimeError(msg, last_err) [ 1010.049632] env[77825]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1010.049632] env[77825]: ERROR neutron [ 1012.637712] env[77912]: INFO neutron.common.config [-] Logging enabled! [ 1012.638002] env[77912]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1012.638409] env[77912]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77912) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1012.638718] env[77912]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1012.638908] env[77912]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1012.639808] env[77912]: WARNING oslo_policy.policy [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1012.640123] env[77912]: WARNING oslo_policy.policy [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1012.641359] env[77912]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1012.656126] env[77912]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Session token - acquiring [ 1012.656940] env[77912]: DEBUG networking_nsxv3.common.locking [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77912) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1012.657203] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1012.657387] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1012.658191] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1012.666867] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1012.667178] env[77912]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1012.667394] env[77912]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77912) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1014.673461] env[77912]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Session token - acquiring [ 1014.673979] env[77912]: DEBUG networking_nsxv3.common.locking [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=77912) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1014.674071] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1014.674217] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1014.674542] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1014.683727] env[77912]: DEBUG oslo_concurrency.lockutils [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77912) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1014.683947] env[77912]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1014.684102] env[77912]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77912) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1016.687069] env[77912]: CRITICAL neutron [None req-2887856f-3f10-491c-aecb-c45a4bb9b752 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1016.687069] env[77912]: ERROR neutron Traceback (most recent call last): [ 1016.687069] env[77912]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1016.687069] env[77912]: ERROR neutron sys.exit(main()) [ 1016.687069] env[77912]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1016.687069] env[77912]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1016.687069] env[77912]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1016.687069] env[77912]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1016.687069] env[77912]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1016.687069] env[77912]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1016.687069] env[77912]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1016.687069] env[77912]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1016.687069] env[77912]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1016.687069] env[77912]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1016.687069] env[77912]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1016.687069] env[77912]: ERROR neutron response = self.get(path=path, params=params) [ 1016.688106] env[77912]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1016.688106] env[77912]: ERROR neutron raise RuntimeError(msg, last_err) [ 1016.688106] env[77912]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1016.688106] env[77912]: ERROR neutron [ 1019.270709] env[78088]: INFO neutron.common.config [-] Logging enabled! [ 1019.271165] env[78088]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1019.271216] env[78088]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78088) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1019.271424] env[78088]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1019.271520] env[78088]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1019.272404] env[78088]: WARNING oslo_policy.policy [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1019.272623] env[78088]: WARNING oslo_policy.policy [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1019.273794] env[78088]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1019.287418] env[78088]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Session token - acquiring [ 1019.288181] env[78088]: DEBUG networking_nsxv3.common.locking [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78088) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1019.288388] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1019.288516] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1019.288871] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1019.297803] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1019.298118] env[78088]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1019.298274] env[78088]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78088) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1021.305880] env[78088]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Session token - acquiring [ 1021.306208] env[78088]: DEBUG networking_nsxv3.common.locking [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78088) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1021.306401] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1021.306527] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1021.306933] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1021.315953] env[78088]: DEBUG oslo_concurrency.lockutils [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78088) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1021.316207] env[78088]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1021.316349] env[78088]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78088) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1023.319250] env[78088]: CRITICAL neutron [None req-f1f08d65-a8c2-4b9b-a96f-463b450068db None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1023.319250] env[78088]: ERROR neutron Traceback (most recent call last): [ 1023.319250] env[78088]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1023.319250] env[78088]: ERROR neutron sys.exit(main()) [ 1023.319250] env[78088]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1023.319250] env[78088]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1023.319250] env[78088]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1023.319250] env[78088]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1023.319250] env[78088]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1023.319250] env[78088]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1023.319250] env[78088]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1023.319250] env[78088]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1023.319250] env[78088]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1023.319250] env[78088]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1023.319250] env[78088]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1023.319250] env[78088]: ERROR neutron response = self.get(path=path, params=params) [ 1023.320162] env[78088]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1023.320162] env[78088]: ERROR neutron raise RuntimeError(msg, last_err) [ 1023.320162] env[78088]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1023.320162] env[78088]: ERROR neutron [ 1025.774511] env[78108]: INFO neutron.common.config [-] Logging enabled! [ 1025.774830] env[78108]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1025.775075] env[78108]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78108) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1025.775293] env[78108]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1025.775452] env[78108]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1025.776226] env[78108]: WARNING oslo_policy.policy [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1025.776456] env[78108]: WARNING oslo_policy.policy [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1025.777672] env[78108]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1025.794546] env[78108]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Session token - acquiring [ 1025.795350] env[78108]: DEBUG networking_nsxv3.common.locking [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78108) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1025.795577] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1025.795709] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1025.796091] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1025.805063] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1025.805307] env[78108]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1025.805474] env[78108]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78108) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1027.811483] env[78108]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Session token - acquiring [ 1027.811843] env[78108]: DEBUG networking_nsxv3.common.locking [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78108) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1027.812069] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1027.812199] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1027.812478] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1027.821865] env[78108]: DEBUG oslo_concurrency.lockutils [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78108) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1027.822100] env[78108]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1027.822236] env[78108]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78108) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1029.825223] env[78108]: CRITICAL neutron [None req-5953d2f0-681d-484d-97e8-e1e277645fc6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1029.825223] env[78108]: ERROR neutron Traceback (most recent call last): [ 1029.825223] env[78108]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1029.825223] env[78108]: ERROR neutron sys.exit(main()) [ 1029.825223] env[78108]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1029.825223] env[78108]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1029.825223] env[78108]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1029.825223] env[78108]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1029.825223] env[78108]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1029.825223] env[78108]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1029.825223] env[78108]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1029.825223] env[78108]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1029.825223] env[78108]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1029.825223] env[78108]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1029.825223] env[78108]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1029.825223] env[78108]: ERROR neutron response = self.get(path=path, params=params) [ 1029.826042] env[78108]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1029.826042] env[78108]: ERROR neutron raise RuntimeError(msg, last_err) [ 1029.826042] env[78108]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1029.826042] env[78108]: ERROR neutron [ 1032.404843] env[78114]: INFO neutron.common.config [-] Logging enabled! [ 1032.405159] env[78114]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1032.405358] env[78114]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78114) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1032.405590] env[78114]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1032.405687] env[78114]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1032.406522] env[78114]: WARNING oslo_policy.policy [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1032.406740] env[78114]: WARNING oslo_policy.policy [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1032.407921] env[78114]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1032.422001] env[78114]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Session token - acquiring [ 1032.422782] env[78114]: DEBUG networking_nsxv3.common.locking [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78114) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1032.422986] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1032.423112] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1032.423471] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1032.432208] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1032.432446] env[78114]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1032.432594] env[78114]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78114) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1034.438572] env[78114]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Session token - acquiring [ 1034.438954] env[78114]: DEBUG networking_nsxv3.common.locking [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78114) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1034.439108] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1034.439238] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1034.439547] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1034.448203] env[78114]: DEBUG oslo_concurrency.lockutils [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78114) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1034.448438] env[78114]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1034.448584] env[78114]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78114) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1036.451134] env[78114]: CRITICAL neutron [None req-917e0141-3878-4ab1-89f5-0d58b0070905 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1036.451134] env[78114]: ERROR neutron Traceback (most recent call last): [ 1036.451134] env[78114]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1036.451134] env[78114]: ERROR neutron sys.exit(main()) [ 1036.451134] env[78114]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1036.451134] env[78114]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1036.451134] env[78114]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1036.451134] env[78114]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1036.451134] env[78114]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1036.451134] env[78114]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1036.451134] env[78114]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1036.451134] env[78114]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1036.451134] env[78114]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1036.451134] env[78114]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1036.451134] env[78114]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1036.451134] env[78114]: ERROR neutron response = self.get(path=path, params=params) [ 1036.452094] env[78114]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1036.452094] env[78114]: ERROR neutron raise RuntimeError(msg, last_err) [ 1036.452094] env[78114]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1036.452094] env[78114]: ERROR neutron [ 1038.755047] env[78185]: INFO neutron.common.config [-] Logging enabled! [ 1038.755427] env[78185]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1038.755539] env[78185]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78185) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1038.755769] env[78185]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1038.755865] env[78185]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1038.756689] env[78185]: WARNING oslo_policy.policy [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1038.756910] env[78185]: WARNING oslo_policy.policy [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1038.758074] env[78185]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1038.772523] env[78185]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Session token - acquiring [ 1038.773284] env[78185]: DEBUG networking_nsxv3.common.locking [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78185) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1038.773482] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1038.773610] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1038.773962] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1038.782644] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1038.782882] env[78185]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1038.783032] env[78185]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78185) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1040.789085] env[78185]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Session token - acquiring [ 1040.789371] env[78185]: DEBUG networking_nsxv3.common.locking [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78185) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1040.789693] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1040.789829] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1040.790102] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1040.799015] env[78185]: DEBUG oslo_concurrency.lockutils [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78185) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1040.799440] env[78185]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1040.799625] env[78185]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78185) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1042.800904] env[78185]: CRITICAL neutron [None req-c02069a4-7779-4de3-980a-54da594c7b94 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1042.800904] env[78185]: ERROR neutron Traceback (most recent call last): [ 1042.800904] env[78185]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1042.800904] env[78185]: ERROR neutron sys.exit(main()) [ 1042.800904] env[78185]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1042.800904] env[78185]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1042.800904] env[78185]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1042.800904] env[78185]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1042.800904] env[78185]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1042.800904] env[78185]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1042.800904] env[78185]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1042.800904] env[78185]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1042.800904] env[78185]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1042.800904] env[78185]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1042.800904] env[78185]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1042.800904] env[78185]: ERROR neutron response = self.get(path=path, params=params) [ 1042.801637] env[78185]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1042.801637] env[78185]: ERROR neutron raise RuntimeError(msg, last_err) [ 1042.801637] env[78185]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1042.801637] env[78185]: ERROR neutron [ 1045.278179] env[78235]: INFO neutron.common.config [-] Logging enabled! [ 1045.278567] env[78235]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1045.278707] env[78235]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78235) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1045.278915] env[78235]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1045.279011] env[78235]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1045.279856] env[78235]: WARNING oslo_policy.policy [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1045.280091] env[78235]: WARNING oslo_policy.policy [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1045.281278] env[78235]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1045.295161] env[78235]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Session token - acquiring [ 1045.295899] env[78235]: DEBUG networking_nsxv3.common.locking [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78235) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1045.296110] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1045.296240] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1045.296599] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1045.305306] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1045.305548] env[78235]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1045.305733] env[78235]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78235) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1047.311964] env[78235]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Session token - acquiring [ 1047.312462] env[78235]: DEBUG networking_nsxv3.common.locking [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78235) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1047.312684] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1047.312818] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1047.313119] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1047.322169] env[78235]: DEBUG oslo_concurrency.lockutils [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78235) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1047.322396] env[78235]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1047.322537] env[78235]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78235) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1049.325457] env[78235]: CRITICAL neutron [None req-ce77b854-b4f4-40da-872d-2ebef7a796cd None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1049.325457] env[78235]: ERROR neutron Traceback (most recent call last): [ 1049.325457] env[78235]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1049.325457] env[78235]: ERROR neutron sys.exit(main()) [ 1049.325457] env[78235]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1049.325457] env[78235]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1049.325457] env[78235]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1049.325457] env[78235]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1049.325457] env[78235]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1049.325457] env[78235]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1049.325457] env[78235]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1049.325457] env[78235]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1049.325457] env[78235]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1049.325457] env[78235]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1049.325457] env[78235]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1049.325457] env[78235]: ERROR neutron response = self.get(path=path, params=params) [ 1049.326297] env[78235]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1049.326297] env[78235]: ERROR neutron raise RuntimeError(msg, last_err) [ 1049.326297] env[78235]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1049.326297] env[78235]: ERROR neutron [ 1051.764588] env[78240]: INFO neutron.common.config [-] Logging enabled! [ 1051.764993] env[78240]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1051.765099] env[78240]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78240) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1051.765299] env[78240]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1051.765391] env[78240]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1051.766193] env[78240]: WARNING oslo_policy.policy [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1051.766417] env[78240]: WARNING oslo_policy.policy [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1051.767576] env[78240]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1051.781580] env[78240]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Session token - acquiring [ 1051.782338] env[78240]: DEBUG networking_nsxv3.common.locking [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78240) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1051.782533] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1051.782662] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1051.783009] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1051.792071] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1051.792317] env[78240]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1051.792462] env[78240]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78240) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1053.798619] env[78240]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Session token - acquiring [ 1053.799111] env[78240]: DEBUG networking_nsxv3.common.locking [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78240) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1053.799311] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1053.799435] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1053.799733] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1053.808624] env[78240]: DEBUG oslo_concurrency.lockutils [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78240) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1053.808851] env[78240]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1053.808988] env[78240]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78240) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1055.811977] env[78240]: CRITICAL neutron [None req-cf5aac4b-783b-4ad4-9dfc-8a166a0a4fcf None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1055.811977] env[78240]: ERROR neutron Traceback (most recent call last): [ 1055.811977] env[78240]: ERROR neutron File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in [ 1055.811977] env[78240]: ERROR neutron sys.exit(main()) [ 1055.811977] env[78240]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 1055.811977] env[78240]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 1055.811977] env[78240]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 1055.811977] env[78240]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 1055.811977] env[78240]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 1055.811977] env[78240]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 1055.811977] env[78240]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 1055.811977] env[78240]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 1055.811977] env[78240]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 1055.811977] env[78240]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 1055.811977] env[78240]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 1055.811977] env[78240]: ERROR neutron response = self.get(path=path, params=params) [ 1055.812805] env[78240]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 1055.812805] env[78240]: ERROR neutron raise RuntimeError(msg, last_err) [ 1055.812805] env[78240]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 1055.812805] env[78240]: ERROR neutron [ 1058.236777] env[78249]: INFO neutron.common.config [-] Logging enabled! [ 1058.237203] env[78249]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 25.0.0.0b2.dev45 [ 1058.237294] env[78249]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78249) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 1058.237638] env[78249]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 1058.237638] env[78249]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 1058.238431] env[78249]: WARNING oslo_policy.policy [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1058.238655] env[78249]: WARNING oslo_policy.policy [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. [ 1058.239845] env[78249]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 1058.253887] env[78249]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Session token - acquiring [ 1058.254628] env[78249]: DEBUG networking_nsxv3.common.locking [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78249) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1058.254824] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1058.254951] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1058.255300] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1058.263933] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1058.264194] env[78249]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1058.264346] env[78249]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78249) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 1060.270217] env[78249]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Session token - acquiring [ 1060.270639] env[78249]: DEBUG networking_nsxv3.common.locking [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=78249) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 1060.270814] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}} [ 1060.270944] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}} [ 1060.271225] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}} [ 1060.280025] env[78249]: DEBUG oslo_concurrency.lockutils [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78249) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}} [ 1060.280265] env[78249]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 1060.280403] env[78249]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-782067e7-4244-4b62-9f26-5bf23fa91adc None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78249) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}