[  541.391405] env[62925]: INFO neutron.common.config [-] Logging enabled!
[  541.392051] env[62925]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  541.392051] env[62925]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62925) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  541.392227] env[62925]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  541.392295] env[62925]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  541.393150] env[62925]: WARNING oslo_policy.policy [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  541.393364] env[62925]: WARNING oslo_policy.policy [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  541.394574] env[62925]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  541.409753] env[62925]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Session token - acquiring
[  541.410538] env[62925]: DEBUG networking_nsxv3.common.locking [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=62925) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.410755] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.410890] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.411431] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.420768] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.421025] env[62925]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  541.421178] env[62925]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62925) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  543.427686] env[62925]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Session token - acquiring
[  543.428063] env[62925]: DEBUG networking_nsxv3.common.locking [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=62925) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.428294] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.428429] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.428707] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.438074] env[62925]: DEBUG oslo_concurrency.lockutils [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62925) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.438308] env[62925]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  543.438452] env[62925]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62925) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  545.441396] env[62925]: CRITICAL neutron [None req-d3b6df15-4544-4c29-9080-7430e203a34a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  545.441396] env[62925]: ERROR neutron Traceback (most recent call last):
[  545.441396] env[62925]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  545.441396] env[62925]: ERROR neutron     sys.exit(main())
[  545.441396] env[62925]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  545.441396] env[62925]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  545.441396] env[62925]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  545.441396] env[62925]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  545.441396] env[62925]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  545.441396] env[62925]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  545.441396] env[62925]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  545.441396] env[62925]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  545.441396] env[62925]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  545.441396] env[62925]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  545.441396] env[62925]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  545.441396] env[62925]: ERROR neutron     response = self.get(path=path, params=params)
[  545.442573] env[62925]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  545.442573] env[62925]: ERROR neutron     raise RuntimeError(msg, last_err)
[  545.442573] env[62925]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  545.442573] env[62925]: ERROR neutron 
[  547.901091] env[62957]: INFO neutron.common.config [-] Logging enabled!
[  547.901507] env[62957]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  547.901678] env[62957]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62957) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  547.901835] env[62957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  547.901930] env[62957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  547.902779] env[62957]: WARNING oslo_policy.policy [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  547.902986] env[62957]: WARNING oslo_policy.policy [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  547.904173] env[62957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  547.918291] env[62957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Session token - acquiring
[  547.919046] env[62957]: DEBUG networking_nsxv3.common.locking [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=62957) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.919241] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.919371] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.919726] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.928651] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.928888] env[62957]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  547.929037] env[62957]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62957) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  549.935173] env[62957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Session token - acquiring
[  549.935563] env[62957]: DEBUG networking_nsxv3.common.locking [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=62957) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  549.935718] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  549.935827] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  549.936146] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  549.945476] env[62957]: DEBUG oslo_concurrency.lockutils [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62957) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  549.945722] env[62957]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  549.945863] env[62957]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62957) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  551.948835] env[62957]: CRITICAL neutron [None req-a3e83174-7be7-4643-8d11-7fc30575b20f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  551.948835] env[62957]: ERROR neutron Traceback (most recent call last):
[  551.948835] env[62957]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  551.948835] env[62957]: ERROR neutron     sys.exit(main())
[  551.948835] env[62957]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  551.948835] env[62957]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  551.948835] env[62957]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  551.948835] env[62957]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  551.948835] env[62957]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  551.948835] env[62957]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  551.948835] env[62957]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  551.948835] env[62957]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  551.948835] env[62957]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  551.948835] env[62957]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  551.948835] env[62957]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  551.948835] env[62957]: ERROR neutron     response = self.get(path=path, params=params)
[  551.949713] env[62957]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  551.949713] env[62957]: ERROR neutron     raise RuntimeError(msg, last_err)
[  551.949713] env[62957]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  551.949713] env[62957]: ERROR neutron 
[  554.428455] env[62981]: INFO neutron.common.config [-] Logging enabled!
[  554.428864] env[62981]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  554.428987] env[62981]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62981) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  554.429200] env[62981]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  554.429295] env[62981]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  554.430121] env[62981]: WARNING oslo_policy.policy [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  554.430334] env[62981]: WARNING oslo_policy.policy [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  554.431525] env[62981]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  554.446537] env[62981]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Session token - acquiring
[  554.447307] env[62981]: DEBUG networking_nsxv3.common.locking [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=62981) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  554.447508] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  554.447636] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  554.447994] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  554.456798] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  554.457038] env[62981]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  554.457193] env[62981]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62981) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  556.463487] env[62981]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Session token - acquiring
[  556.463924] env[62981]: DEBUG networking_nsxv3.common.locking [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=62981) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  556.464030] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  556.464071] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  556.464539] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  556.475265] env[62981]: DEBUG oslo_concurrency.lockutils [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62981) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  556.475265] env[62981]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  556.475265] env[62981]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62981) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  558.476982] env[62981]: CRITICAL neutron [None req-f6042423-d4a3-47fe-850e-42a6c135a5aa None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  558.476982] env[62981]: ERROR neutron Traceback (most recent call last):
[  558.476982] env[62981]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  558.476982] env[62981]: ERROR neutron     sys.exit(main())
[  558.476982] env[62981]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  558.476982] env[62981]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  558.476982] env[62981]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  558.476982] env[62981]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  558.476982] env[62981]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  558.476982] env[62981]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  558.476982] env[62981]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  558.476982] env[62981]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  558.476982] env[62981]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  558.476982] env[62981]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  558.476982] env[62981]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  558.476982] env[62981]: ERROR neutron     response = self.get(path=path, params=params)
[  558.478056] env[62981]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  558.478056] env[62981]: ERROR neutron     raise RuntimeError(msg, last_err)
[  558.478056] env[62981]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  558.478056] env[62981]: ERROR neutron 
[  560.870146] env[63068]: INFO neutron.common.config [-] Logging enabled!
[  560.870541] env[63068]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  560.870654] env[63068]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63068) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  560.870877] env[63068]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  560.870970] env[63068]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  560.871814] env[63068]: WARNING oslo_policy.policy [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  560.872094] env[63068]: WARNING oslo_policy.policy [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  560.873260] env[63068]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  560.887770] env[63068]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Session token - acquiring
[  560.888554] env[63068]: DEBUG networking_nsxv3.common.locking [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63068) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  560.888759] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  560.888887] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  560.889248] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  560.898202] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  560.898441] env[63068]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  560.898593] env[63068]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63068) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  562.904770] env[63068]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Session token - acquiring
[  562.905113] env[63068]: DEBUG networking_nsxv3.common.locking [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63068) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.905369] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.905491] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.905762] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.914737] env[63068]: DEBUG oslo_concurrency.lockutils [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63068) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  562.914962] env[63068]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  562.915099] env[63068]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63068) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  564.917338] env[63068]: CRITICAL neutron [None req-ba766b4e-ccdd-4b6a-b1ae-80d4e63116d5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  564.917338] env[63068]: ERROR neutron Traceback (most recent call last):
[  564.917338] env[63068]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  564.917338] env[63068]: ERROR neutron     sys.exit(main())
[  564.917338] env[63068]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  564.917338] env[63068]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  564.917338] env[63068]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  564.917338] env[63068]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  564.917338] env[63068]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  564.917338] env[63068]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  564.917338] env[63068]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  564.917338] env[63068]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  564.917338] env[63068]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  564.917338] env[63068]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  564.917338] env[63068]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  564.917338] env[63068]: ERROR neutron     response = self.get(path=path, params=params)
[  564.918112] env[63068]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  564.918112] env[63068]: ERROR neutron     raise RuntimeError(msg, last_err)
[  564.918112] env[63068]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  564.918112] env[63068]: ERROR neutron 
[  567.344496] env[63078]: INFO neutron.common.config [-] Logging enabled!
[  567.344972] env[63078]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  567.345042] env[63078]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63078) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  567.345225] env[63078]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  567.345320] env[63078]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  567.346144] env[63078]: WARNING oslo_policy.policy [None req-603684ce-9d00-4156-b924-295d921219de None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  567.346357] env[63078]: WARNING oslo_policy.policy [None req-603684ce-9d00-4156-b924-295d921219de None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  567.347526] env[63078]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-603684ce-9d00-4156-b924-295d921219de None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  567.361100] env[63078]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-603684ce-9d00-4156-b924-295d921219de None None] Session token - acquiring
[  567.361891] env[63078]: DEBUG networking_nsxv3.common.locking [None req-603684ce-9d00-4156-b924-295d921219de None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63078) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.362122] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.362270] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.362571] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.371514] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.371759] env[63078]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-603684ce-9d00-4156-b924-295d921219de None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  567.371910] env[63078]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-603684ce-9d00-4156-b924-295d921219de None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63078) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  569.378163] env[63078]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-603684ce-9d00-4156-b924-295d921219de None None] Session token - acquiring
[  569.378621] env[63078]: DEBUG networking_nsxv3.common.locking [None req-603684ce-9d00-4156-b924-295d921219de None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63078) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  569.378770] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  569.378906] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  569.379185] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  569.388535] env[63078]: DEBUG oslo_concurrency.lockutils [None req-603684ce-9d00-4156-b924-295d921219de None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63078) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  569.388775] env[63078]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-603684ce-9d00-4156-b924-295d921219de None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  569.388917] env[63078]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-603684ce-9d00-4156-b924-295d921219de None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63078) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  571.391845] env[63078]: CRITICAL neutron [None req-603684ce-9d00-4156-b924-295d921219de None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  571.391845] env[63078]: ERROR neutron Traceback (most recent call last):
[  571.391845] env[63078]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  571.391845] env[63078]: ERROR neutron     sys.exit(main())
[  571.391845] env[63078]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  571.391845] env[63078]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  571.391845] env[63078]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  571.391845] env[63078]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  571.391845] env[63078]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  571.391845] env[63078]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  571.391845] env[63078]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  571.391845] env[63078]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  571.391845] env[63078]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  571.391845] env[63078]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  571.391845] env[63078]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  571.391845] env[63078]: ERROR neutron     response = self.get(path=path, params=params)
[  571.393297] env[63078]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  571.393297] env[63078]: ERROR neutron     raise RuntimeError(msg, last_err)
[  571.393297] env[63078]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  571.393297] env[63078]: ERROR neutron 
[  573.880732] env[63084]: INFO neutron.common.config [-] Logging enabled!
[  573.881221] env[63084]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  573.881289] env[63084]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63084) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  573.881519] env[63084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  573.881612] env[63084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  573.882428] env[63084]: WARNING oslo_policy.policy [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  573.882649] env[63084]: WARNING oslo_policy.policy [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  573.883854] env[63084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  573.898105] env[63084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Session token - acquiring
[  573.898825] env[63084]: DEBUG networking_nsxv3.common.locking [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63084) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.899020] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.899147] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.899495] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.908406] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.908801] env[63084]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  573.908959] env[63084]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63084) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  575.915156] env[63084]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Session token - acquiring
[  575.915596] env[63084]: DEBUG networking_nsxv3.common.locking [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63084) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  575.915676] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  575.915778] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  575.916073] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  575.925135] env[63084]: DEBUG oslo_concurrency.lockutils [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63084) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.925385] env[63084]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  575.925522] env[63084]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63084) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  577.928490] env[63084]: CRITICAL neutron [None req-796cbc51-00ed-4914-9651-b42d7dc72fd4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  577.928490] env[63084]: ERROR neutron Traceback (most recent call last):
[  577.928490] env[63084]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  577.928490] env[63084]: ERROR neutron     sys.exit(main())
[  577.928490] env[63084]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  577.928490] env[63084]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  577.928490] env[63084]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  577.928490] env[63084]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  577.928490] env[63084]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  577.928490] env[63084]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  577.928490] env[63084]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  577.928490] env[63084]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  577.928490] env[63084]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  577.928490] env[63084]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  577.928490] env[63084]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  577.928490] env[63084]: ERROR neutron     response = self.get(path=path, params=params)
[  577.929282] env[63084]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  577.929282] env[63084]: ERROR neutron     raise RuntimeError(msg, last_err)
[  577.929282] env[63084]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  577.929282] env[63084]: ERROR neutron 
[  580.716894] env[63189]: INFO neutron.common.config [-] Logging enabled!
[  580.717279] env[63189]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  580.717394] env[63189]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63189) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  580.717604] env[63189]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  580.717695] env[63189]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  580.718571] env[63189]: WARNING oslo_policy.policy [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  580.718810] env[63189]: WARNING oslo_policy.policy [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  580.720067] env[63189]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  580.735480] env[63189]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Session token - acquiring
[  580.736351] env[63189]: DEBUG networking_nsxv3.common.locking [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63189) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.736562] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.736691] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.737064] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.746910] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.747171] env[63189]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  580.747317] env[63189]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63189) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  582.753963] env[63189]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Session token - acquiring
[  582.754345] env[63189]: DEBUG networking_nsxv3.common.locking [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63189) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.754574] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.754668] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.754981] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.764395] env[63189]: DEBUG oslo_concurrency.lockutils [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63189) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.764645] env[63189]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  582.764839] env[63189]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63189) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  584.767808] env[63189]: CRITICAL neutron [None req-63a1f729-5f3b-4fd2-beca-04434a121da3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  584.767808] env[63189]: ERROR neutron Traceback (most recent call last):
[  584.767808] env[63189]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  584.767808] env[63189]: ERROR neutron     sys.exit(main())
[  584.767808] env[63189]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  584.767808] env[63189]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  584.767808] env[63189]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  584.767808] env[63189]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  584.767808] env[63189]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  584.767808] env[63189]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  584.767808] env[63189]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  584.767808] env[63189]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  584.767808] env[63189]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  584.767808] env[63189]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  584.767808] env[63189]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  584.767808] env[63189]: ERROR neutron     response = self.get(path=path, params=params)
[  584.768571] env[63189]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  584.768571] env[63189]: ERROR neutron     raise RuntimeError(msg, last_err)
[  584.768571] env[63189]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  584.768571] env[63189]: ERROR neutron 
[  587.241389] env[63516]: INFO neutron.common.config [-] Logging enabled!
[  587.241719] env[63516]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  587.241912] env[63516]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63516) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  587.242217] env[63516]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  587.242317] env[63516]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  587.243142] env[63516]: WARNING oslo_policy.policy [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  587.243363] env[63516]: WARNING oslo_policy.policy [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  587.244634] env[63516]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  587.258447] env[63516]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Session token - acquiring
[  587.259237] env[63516]: DEBUG networking_nsxv3.common.locking [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63516) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.259443] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.259569] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.259935] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.269078] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.269328] env[63516]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  587.269482] env[63516]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63516) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  589.279690] env[63516]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Session token - acquiring
[  589.279690] env[63516]: DEBUG networking_nsxv3.common.locking [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63516) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.279690] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.280188] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.280188] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.287670] env[63516]: DEBUG oslo_concurrency.lockutils [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63516) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.288245] env[63516]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  589.288508] env[63516]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63516) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  591.292333] env[63516]: CRITICAL neutron [None req-e8dc8c26-f053-42d0-9a9b-8e7d549c63ff None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  591.292333] env[63516]: ERROR neutron Traceback (most recent call last):
[  591.292333] env[63516]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  591.292333] env[63516]: ERROR neutron     sys.exit(main())
[  591.292333] env[63516]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  591.292333] env[63516]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  591.292333] env[63516]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  591.292333] env[63516]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  591.292333] env[63516]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  591.292333] env[63516]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  591.292333] env[63516]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  591.292333] env[63516]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  591.292333] env[63516]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  591.292333] env[63516]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  591.292333] env[63516]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  591.292333] env[63516]: ERROR neutron     response = self.get(path=path, params=params)
[  591.292974] env[63516]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  591.292974] env[63516]: ERROR neutron     raise RuntimeError(msg, last_err)
[  591.292974] env[63516]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  591.292974] env[63516]: ERROR neutron 
[  594.258769] env[63835]: INFO neutron.common.config [-] Logging enabled!
[  594.259026] env[63835]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  594.259342] env[63835]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63835) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  594.259623] env[63835]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  594.259726] env[63835]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  594.260605] env[63835]: WARNING oslo_policy.policy [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  594.260816] env[63835]: WARNING oslo_policy.policy [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  594.262068] env[63835]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  594.293000] env[63835]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Session token - acquiring
[  594.293849] env[63835]: DEBUG networking_nsxv3.common.locking [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63835) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.294066] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.294185] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.294637] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.311885] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.312191] env[63835]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  594.312339] env[63835]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63835) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  596.321085] env[63835]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Session token - acquiring
[  596.321085] env[63835]: DEBUG networking_nsxv3.common.locking [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=63835) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.321085] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.321449] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.321449] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.330034] env[63835]: DEBUG oslo_concurrency.lockutils [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63835) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.330301] env[63835]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  596.330794] env[63835]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63835) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  598.332901] env[63835]: CRITICAL neutron [None req-86e591f1-077a-4d7f-89ac-1831a4507a6b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  598.332901] env[63835]: ERROR neutron Traceback (most recent call last):
[  598.332901] env[63835]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  598.332901] env[63835]: ERROR neutron     sys.exit(main())
[  598.332901] env[63835]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  598.332901] env[63835]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  598.332901] env[63835]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  598.332901] env[63835]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  598.332901] env[63835]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  598.332901] env[63835]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  598.332901] env[63835]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  598.332901] env[63835]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  598.332901] env[63835]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  598.332901] env[63835]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  598.332901] env[63835]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  598.332901] env[63835]: ERROR neutron     response = self.get(path=path, params=params)
[  598.333687] env[63835]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  598.333687] env[63835]: ERROR neutron     raise RuntimeError(msg, last_err)
[  598.333687] env[63835]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  598.333687] env[63835]: ERROR neutron 
[  601.202789] env[64569]: INFO neutron.common.config [-] Logging enabled!
[  601.203078] env[64569]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  601.203237] env[64569]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64569) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  601.203449] env[64569]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  601.203598] env[64569]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  601.204508] env[64569]: WARNING oslo_policy.policy [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  601.204771] env[64569]: WARNING oslo_policy.policy [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  601.206018] env[64569]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  601.221964] env[64569]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Session token - acquiring
[  601.221964] env[64569]: DEBUG networking_nsxv3.common.locking [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=64569) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  601.221964] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  601.222112] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  601.222283] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  601.236395] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  601.236395] env[64569]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  601.236395] env[64569]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64569) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  603.244989] env[64569]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Session token - acquiring
[  603.245292] env[64569]: DEBUG networking_nsxv3.common.locking [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=64569) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  603.245537] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  603.245676] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  603.245944] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  603.255731] env[64569]: DEBUG oslo_concurrency.lockutils [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64569) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  603.255990] env[64569]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  603.256173] env[64569]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64569) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  605.259043] env[64569]: CRITICAL neutron [None req-e9546947-f8fc-4bfc-b758-d2e57670f9d1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  605.259043] env[64569]: ERROR neutron Traceback (most recent call last):
[  605.259043] env[64569]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  605.259043] env[64569]: ERROR neutron     sys.exit(main())
[  605.259043] env[64569]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  605.259043] env[64569]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  605.259043] env[64569]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  605.259043] env[64569]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  605.259043] env[64569]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  605.259043] env[64569]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  605.259043] env[64569]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  605.259043] env[64569]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  605.259043] env[64569]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  605.259043] env[64569]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  605.259043] env[64569]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  605.259043] env[64569]: ERROR neutron     response = self.get(path=path, params=params)
[  605.259776] env[64569]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  605.259776] env[64569]: ERROR neutron     raise RuntimeError(msg, last_err)
[  605.259776] env[64569]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  605.259776] env[64569]: ERROR neutron 
[  608.038830] env[64963]: INFO neutron.common.config [-] Logging enabled!
[  608.039104] env[64963]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  608.039378] env[64963]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64963) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  608.039602] env[64963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  608.039693] env[64963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  608.040579] env[64963]: WARNING oslo_policy.policy [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  608.041103] env[64963]: WARNING oslo_policy.policy [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  608.042028] env[64963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  608.056456] env[64963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Session token - acquiring
[  608.057340] env[64963]: DEBUG networking_nsxv3.common.locking [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=64963) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.057549] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.057673] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.058040] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.067634] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.067907] env[64963]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  608.068062] env[64963]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64963) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  610.076323] env[64963]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Session token - acquiring
[  610.076323] env[64963]: DEBUG networking_nsxv3.common.locking [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=64963) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.076323] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.076658] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.076658] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.088651] env[64963]: DEBUG oslo_concurrency.lockutils [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64963) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.088904] env[64963]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  610.089112] env[64963]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64963) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  612.092188] env[64963]: CRITICAL neutron [None req-c53f8e3a-38c5-44ab-90b9-21e5e773f472 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  612.092188] env[64963]: ERROR neutron Traceback (most recent call last):
[  612.092188] env[64963]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  612.092188] env[64963]: ERROR neutron     sys.exit(main())
[  612.092188] env[64963]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  612.092188] env[64963]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  612.092188] env[64963]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  612.092188] env[64963]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  612.092188] env[64963]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  612.092188] env[64963]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  612.092188] env[64963]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  612.092188] env[64963]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  612.092188] env[64963]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  612.092188] env[64963]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  612.092188] env[64963]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  612.092188] env[64963]: ERROR neutron     response = self.get(path=path, params=params)
[  612.092901] env[64963]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  612.092901] env[64963]: ERROR neutron     raise RuntimeError(msg, last_err)
[  612.092901] env[64963]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  612.092901] env[64963]: ERROR neutron 
[  615.176567] env[65278]: INFO neutron.common.config [-] Logging enabled!
[  615.176821] env[65278]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  615.177393] env[65278]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65278) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  615.177393] env[65278]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  615.177655] env[65278]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  615.178603] env[65278]: WARNING oslo_policy.policy [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  615.182016] env[65278]: WARNING oslo_policy.policy [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  615.182016] env[65278]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  615.203683] env[65278]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Session token - acquiring
[  615.205198] env[65278]: DEBUG networking_nsxv3.common.locking [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=65278) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.205198] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.205392] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.205809] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.221183] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.222933] env[65278]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  615.222933] env[65278]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65278) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  617.228337] env[65278]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Session token - acquiring
[  617.228684] env[65278]: DEBUG networking_nsxv3.common.locking [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=65278) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.228824] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.228995] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.229235] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.238965] env[65278]: DEBUG oslo_concurrency.lockutils [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65278) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.239223] env[65278]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  617.239371] env[65278]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65278) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  619.241164] env[65278]: CRITICAL neutron [None req-33ed2765-e180-4bca-ba22-f7a83543fc80 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  619.241164] env[65278]: ERROR neutron Traceback (most recent call last):
[  619.241164] env[65278]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  619.241164] env[65278]: ERROR neutron     sys.exit(main())
[  619.241164] env[65278]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  619.241164] env[65278]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  619.241164] env[65278]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  619.241164] env[65278]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  619.241164] env[65278]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  619.241164] env[65278]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  619.241164] env[65278]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  619.241164] env[65278]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  619.241164] env[65278]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  619.241164] env[65278]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  619.241164] env[65278]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  619.241164] env[65278]: ERROR neutron     response = self.get(path=path, params=params)
[  619.241974] env[65278]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  619.241974] env[65278]: ERROR neutron     raise RuntimeError(msg, last_err)
[  619.241974] env[65278]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  619.241974] env[65278]: ERROR neutron 
[  622.413251] env[65715]: INFO neutron.common.config [-] Logging enabled!
[  622.413591] env[65715]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  622.413943] env[65715]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65715) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  622.414254] env[65715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  622.414399] env[65715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  622.415379] env[65715]: WARNING oslo_policy.policy [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  622.415682] env[65715]: WARNING oslo_policy.policy [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  622.417286] env[65715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  622.434321] env[65715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Session token - acquiring
[  622.435299] env[65715]: DEBUG networking_nsxv3.common.locking [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=65715) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.435774] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.435846] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.436277] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.446734] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  622.447066] env[65715]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  622.447680] env[65715]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65715) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  624.453754] env[65715]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Session token - acquiring
[  624.454099] env[65715]: DEBUG networking_nsxv3.common.locking [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=65715) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.454274] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.454543] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.454757] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.465378] env[65715]: DEBUG oslo_concurrency.lockutils [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65715) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  624.465378] env[65715]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  624.465601] env[65715]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65715) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  626.470158] env[65715]: CRITICAL neutron [None req-be877ab4-4a47-491b-861a-70a4aeff713f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  626.470158] env[65715]: ERROR neutron Traceback (most recent call last):
[  626.470158] env[65715]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  626.470158] env[65715]: ERROR neutron     sys.exit(main())
[  626.470158] env[65715]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  626.470158] env[65715]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  626.470158] env[65715]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  626.470158] env[65715]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  626.470158] env[65715]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  626.470158] env[65715]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  626.470158] env[65715]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  626.470158] env[65715]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  626.470158] env[65715]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  626.470158] env[65715]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  626.470158] env[65715]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  626.470158] env[65715]: ERROR neutron     response = self.get(path=path, params=params)
[  626.471031] env[65715]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  626.471031] env[65715]: ERROR neutron     raise RuntimeError(msg, last_err)
[  626.471031] env[65715]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  626.471031] env[65715]: ERROR neutron 
[  629.222655] env[66261]: INFO neutron.common.config [-] Logging enabled!
[  629.222655] env[66261]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  629.222655] env[66261]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66261) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  629.222655] env[66261]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  629.222655] env[66261]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  629.223246] env[66261]: WARNING oslo_policy.policy [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  629.223246] env[66261]: WARNING oslo_policy.policy [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  629.224769] env[66261]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  629.240502] env[66261]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Session token - acquiring
[  629.241678] env[66261]: DEBUG networking_nsxv3.common.locking [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=66261) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  629.241893] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  629.242146] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  629.242449] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  629.255914] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  629.256217] env[66261]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  629.256363] env[66261]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66261) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  631.267222] env[66261]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Session token - acquiring
[  631.267606] env[66261]: DEBUG networking_nsxv3.common.locking [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=66261) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  631.267871] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  631.267950] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  631.268258] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  631.277786] env[66261]: DEBUG oslo_concurrency.lockutils [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66261) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  631.278065] env[66261]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  631.278172] env[66261]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66261) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  633.280885] env[66261]: CRITICAL neutron [None req-de3e8286-df5f-483a-9774-efb7371868cf None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  633.280885] env[66261]: ERROR neutron Traceback (most recent call last):
[  633.280885] env[66261]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  633.280885] env[66261]: ERROR neutron     sys.exit(main())
[  633.280885] env[66261]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  633.280885] env[66261]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  633.280885] env[66261]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  633.280885] env[66261]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  633.280885] env[66261]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  633.280885] env[66261]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  633.280885] env[66261]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  633.280885] env[66261]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  633.280885] env[66261]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  633.280885] env[66261]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  633.280885] env[66261]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  633.280885] env[66261]: ERROR neutron     response = self.get(path=path, params=params)
[  633.281633] env[66261]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  633.281633] env[66261]: ERROR neutron     raise RuntimeError(msg, last_err)
[  633.281633] env[66261]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  633.281633] env[66261]: ERROR neutron 
[  636.189779] env[66783]: INFO neutron.common.config [-] Logging enabled!
[  636.190075] env[66783]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  636.190405] env[66783]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66783) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  636.190650] env[66783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  636.190795] env[66783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  636.191649] env[66783]: WARNING oslo_policy.policy [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  636.191871] env[66783]: WARNING oslo_policy.policy [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  636.193717] env[66783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  636.208529] env[66783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Session token - acquiring
[  636.209912] env[66783]: DEBUG networking_nsxv3.common.locking [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=66783) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  636.209912] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  636.210078] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  636.210260] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  636.220167] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  636.220456] env[66783]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  636.220673] env[66783]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66783) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  638.227219] env[66783]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Session token - acquiring
[  638.227219] env[66783]: DEBUG networking_nsxv3.common.locking [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=66783) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.227576] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.227576] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.227785] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.237444] env[66783]: DEBUG oslo_concurrency.lockutils [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66783) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.237693] env[66783]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  638.237828] env[66783]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66783) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  640.238842] env[66783]: CRITICAL neutron [None req-9bab2e9e-93eb-4b64-b654-8db2b845e377 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  640.238842] env[66783]: ERROR neutron Traceback (most recent call last):
[  640.238842] env[66783]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  640.238842] env[66783]: ERROR neutron     sys.exit(main())
[  640.238842] env[66783]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  640.238842] env[66783]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  640.238842] env[66783]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  640.238842] env[66783]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  640.238842] env[66783]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  640.238842] env[66783]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  640.238842] env[66783]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  640.238842] env[66783]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  640.238842] env[66783]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  640.238842] env[66783]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  640.238842] env[66783]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  640.238842] env[66783]: ERROR neutron     response = self.get(path=path, params=params)
[  640.239668] env[66783]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  640.239668] env[66783]: ERROR neutron     raise RuntimeError(msg, last_err)
[  640.239668] env[66783]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  640.239668] env[66783]: ERROR neutron 
[  643.262609] env[67315]: INFO neutron.common.config [-] Logging enabled!
[  643.262980] env[67315]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  643.263196] env[67315]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67315) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  643.263444] env[67315]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  643.263535] env[67315]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  643.264538] env[67315]: WARNING oslo_policy.policy [None req-a851d858-9879-4000-a375-865e91650a55 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  643.264773] env[67315]: WARNING oslo_policy.policy [None req-a851d858-9879-4000-a375-865e91650a55 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  643.266430] env[67315]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a851d858-9879-4000-a375-865e91650a55 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  643.281527] env[67315]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a851d858-9879-4000-a375-865e91650a55 None None] Session token - acquiring
[  643.282551] env[67315]: DEBUG networking_nsxv3.common.locking [None req-a851d858-9879-4000-a375-865e91650a55 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=67315) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.282765] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.282903] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.283295] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.292871] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.293109] env[67315]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a851d858-9879-4000-a375-865e91650a55 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  643.293301] env[67315]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a851d858-9879-4000-a375-865e91650a55 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67315) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  645.299654] env[67315]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a851d858-9879-4000-a375-865e91650a55 None None] Session token - acquiring
[  645.300556] env[67315]: DEBUG networking_nsxv3.common.locking [None req-a851d858-9879-4000-a375-865e91650a55 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=67315) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.300556] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.300767] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.300804] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.311772] env[67315]: DEBUG oslo_concurrency.lockutils [None req-a851d858-9879-4000-a375-865e91650a55 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67315) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.312041] env[67315]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a851d858-9879-4000-a375-865e91650a55 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  645.312223] env[67315]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a851d858-9879-4000-a375-865e91650a55 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67315) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  647.315197] env[67315]: CRITICAL neutron [None req-a851d858-9879-4000-a375-865e91650a55 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  647.315197] env[67315]: ERROR neutron Traceback (most recent call last):
[  647.315197] env[67315]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  647.315197] env[67315]: ERROR neutron     sys.exit(main())
[  647.315197] env[67315]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  647.315197] env[67315]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  647.315197] env[67315]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  647.315197] env[67315]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  647.315197] env[67315]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  647.315197] env[67315]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  647.315197] env[67315]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  647.315197] env[67315]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  647.315197] env[67315]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  647.315197] env[67315]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  647.315197] env[67315]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  647.315197] env[67315]: ERROR neutron     response = self.get(path=path, params=params)
[  647.316083] env[67315]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  647.316083] env[67315]: ERROR neutron     raise RuntimeError(msg, last_err)
[  647.316083] env[67315]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  647.316083] env[67315]: ERROR neutron 
[  650.070245] env[67774]: INFO neutron.common.config [-] Logging enabled!
[  650.070572] env[67774]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  650.070931] env[67774]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67774) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  650.071050] env[67774]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  650.071151] env[67774]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  650.072091] env[67774]: WARNING oslo_policy.policy [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  650.072245] env[67774]: WARNING oslo_policy.policy [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  650.073488] env[67774]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  650.088334] env[67774]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Session token - acquiring
[  650.089290] env[67774]: DEBUG networking_nsxv3.common.locking [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=67774) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.089546] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.089744] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.090121] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.100070] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.100389] env[67774]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  650.100567] env[67774]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67774) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  652.107130] env[67774]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Session token - acquiring
[  652.107440] env[67774]: DEBUG networking_nsxv3.common.locking [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=67774) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.107747] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.107747] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.107997] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.120435] env[67774]: DEBUG oslo_concurrency.lockutils [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67774) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.120435] env[67774]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  652.120435] env[67774]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67774) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  654.128306] env[67774]: CRITICAL neutron [None req-718582bf-afff-4daa-bbbc-182fb1de6208 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  654.128306] env[67774]: ERROR neutron Traceback (most recent call last):
[  654.128306] env[67774]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  654.128306] env[67774]: ERROR neutron     sys.exit(main())
[  654.128306] env[67774]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  654.128306] env[67774]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  654.128306] env[67774]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  654.128306] env[67774]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  654.128306] env[67774]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  654.128306] env[67774]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  654.128306] env[67774]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  654.128306] env[67774]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  654.128306] env[67774]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  654.128306] env[67774]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  654.128306] env[67774]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  654.128306] env[67774]: ERROR neutron     response = self.get(path=path, params=params)
[  654.128999] env[67774]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  654.128999] env[67774]: ERROR neutron     raise RuntimeError(msg, last_err)
[  654.128999] env[67774]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  654.128999] env[67774]: ERROR neutron 
[  656.875780] env[68300]: INFO neutron.common.config [-] Logging enabled!
[  656.875780] env[68300]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  656.876159] env[68300]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68300) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  656.876159] env[68300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  656.876264] env[68300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  656.877153] env[68300]: WARNING oslo_policy.policy [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  656.877365] env[68300]: WARNING oslo_policy.policy [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  656.878621] env[68300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  656.901336] env[68300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Session token - acquiring
[  656.902169] env[68300]: DEBUG networking_nsxv3.common.locking [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=68300) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.902379] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.902500] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.902976] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.913557] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.913824] env[68300]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  656.913976] env[68300]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68300) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  658.923101] env[68300]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Session token - acquiring
[  658.923101] env[68300]: DEBUG networking_nsxv3.common.locking [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=68300) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.923101] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.923467] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.923467] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.945182] env[68300]: DEBUG oslo_concurrency.lockutils [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68300) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.945182] env[68300]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  658.945182] env[68300]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68300) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  660.947857] env[68300]: CRITICAL neutron [None req-ae1f5b1a-d708-488f-88db-dcff70f1c463 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  660.947857] env[68300]: ERROR neutron Traceback (most recent call last):
[  660.947857] env[68300]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  660.947857] env[68300]: ERROR neutron     sys.exit(main())
[  660.947857] env[68300]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  660.947857] env[68300]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  660.947857] env[68300]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  660.947857] env[68300]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  660.947857] env[68300]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  660.947857] env[68300]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  660.947857] env[68300]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  660.947857] env[68300]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  660.947857] env[68300]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  660.947857] env[68300]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  660.947857] env[68300]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  660.947857] env[68300]: ERROR neutron     response = self.get(path=path, params=params)
[  660.948688] env[68300]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  660.948688] env[68300]: ERROR neutron     raise RuntimeError(msg, last_err)
[  660.948688] env[68300]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  660.948688] env[68300]: ERROR neutron 
[  664.004461] env[68589]: INFO neutron.common.config [-] Logging enabled!
[  664.004950] env[68589]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  664.005089] env[68589]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68589) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  664.005336] env[68589]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  664.005434] env[68589]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  664.006719] env[68589]: WARNING oslo_policy.policy [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  664.006882] env[68589]: WARNING oslo_policy.policy [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  664.008260] env[68589]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  664.027400] env[68589]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Session token - acquiring
[  664.027400] env[68589]: DEBUG networking_nsxv3.common.locking [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=68589) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.027400] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.027981] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.027981] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.036928] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.037354] env[68589]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  664.037630] env[68589]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68589) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  666.045035] env[68589]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Session token - acquiring
[  666.045035] env[68589]: DEBUG networking_nsxv3.common.locking [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=68589) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.045035] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.045425] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.045425] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.061071] env[68589]: DEBUG oslo_concurrency.lockutils [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68589) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.061344] env[68589]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  666.061482] env[68589]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68589) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  668.064552] env[68589]: CRITICAL neutron [None req-b22a0694-51b4-4d0d-8147-f0936595a345 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  668.064552] env[68589]: ERROR neutron Traceback (most recent call last):
[  668.064552] env[68589]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  668.064552] env[68589]: ERROR neutron     sys.exit(main())
[  668.064552] env[68589]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  668.064552] env[68589]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  668.064552] env[68589]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  668.064552] env[68589]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  668.064552] env[68589]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  668.064552] env[68589]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  668.064552] env[68589]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  668.064552] env[68589]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  668.064552] env[68589]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  668.064552] env[68589]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  668.064552] env[68589]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  668.064552] env[68589]: ERROR neutron     response = self.get(path=path, params=params)
[  668.066175] env[68589]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  668.066175] env[68589]: ERROR neutron     raise RuntimeError(msg, last_err)
[  668.066175] env[68589]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  668.066175] env[68589]: ERROR neutron 
[  670.945759] env[69000]: INFO neutron.common.config [-] Logging enabled!
[  670.945759] env[69000]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  670.945759] env[69000]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69000) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  670.945759] env[69000]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  670.945759] env[69000]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  670.945759] env[69000]: WARNING oslo_policy.policy [None req-52717b40-486a-45a9-b963-085a48e37275 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  670.945759] env[69000]: WARNING oslo_policy.policy [None req-52717b40-486a-45a9-b963-085a48e37275 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  670.945759] env[69000]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  670.946599] env[69000]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Session token - acquiring
[  670.946599] env[69000]: DEBUG networking_nsxv3.common.locking [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=69000) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.946599] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.947159] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.947159] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.953617] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.953896] env[69000]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  670.954050] env[69000]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69000) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  672.963952] env[69000]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Session token - acquiring
[  672.963952] env[69000]: DEBUG networking_nsxv3.common.locking [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=69000) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.963952] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.964423] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.964423] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.975874] env[69000]: DEBUG oslo_concurrency.lockutils [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69000) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.976172] env[69000]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  672.976324] env[69000]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69000) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  674.977693] env[69000]: CRITICAL neutron [None req-52717b40-486a-45a9-b963-085a48e37275 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  674.977693] env[69000]: ERROR neutron Traceback (most recent call last):
[  674.977693] env[69000]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  674.977693] env[69000]: ERROR neutron     sys.exit(main())
[  674.977693] env[69000]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  674.977693] env[69000]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  674.977693] env[69000]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  674.977693] env[69000]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  674.977693] env[69000]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  674.977693] env[69000]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  674.977693] env[69000]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  674.977693] env[69000]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  674.977693] env[69000]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  674.977693] env[69000]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  674.977693] env[69000]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  674.977693] env[69000]: ERROR neutron     response = self.get(path=path, params=params)
[  674.978368] env[69000]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  674.978368] env[69000]: ERROR neutron     raise RuntimeError(msg, last_err)
[  674.978368] env[69000]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  674.978368] env[69000]: ERROR neutron 
[  678.046747] env[69342]: INFO neutron.common.config [-] Logging enabled!
[  678.047011] env[69342]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  678.047382] env[69342]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69342) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  678.048139] env[69342]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  678.048139] env[69342]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  678.048722] env[69342]: WARNING oslo_policy.policy [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  678.048948] env[69342]: WARNING oslo_policy.policy [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  678.050293] env[69342]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  678.066020] env[69342]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Session token - acquiring
[  678.068347] env[69342]: DEBUG networking_nsxv3.common.locking [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=69342) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.068347] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.068520] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.068520] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.078139] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.078423] env[69342]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  678.078578] env[69342]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69342) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  680.086131] env[69342]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Session token - acquiring
[  680.086458] env[69342]: DEBUG networking_nsxv3.common.locking [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=69342) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.086687] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.086828] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.087117] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.096388] env[69342]: DEBUG oslo_concurrency.lockutils [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69342) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.096630] env[69342]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  680.096765] env[69342]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69342) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  682.104031] env[69342]: CRITICAL neutron [None req-76a5979a-166c-4473-8aea-519736fa3852 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  682.104031] env[69342]: ERROR neutron Traceback (most recent call last):
[  682.104031] env[69342]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  682.104031] env[69342]: ERROR neutron     sys.exit(main())
[  682.104031] env[69342]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  682.104031] env[69342]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  682.104031] env[69342]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  682.104031] env[69342]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  682.104031] env[69342]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  682.104031] env[69342]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  682.104031] env[69342]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  682.104031] env[69342]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  682.104031] env[69342]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  682.104031] env[69342]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  682.104031] env[69342]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  682.104031] env[69342]: ERROR neutron     response = self.get(path=path, params=params)
[  682.105007] env[69342]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  682.105007] env[69342]: ERROR neutron     raise RuntimeError(msg, last_err)
[  682.105007] env[69342]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  682.105007] env[69342]: ERROR neutron 
[  684.928759] env[69702]: INFO neutron.common.config [-] Logging enabled!
[  684.929018] env[69702]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  684.929639] env[69702]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69702) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  684.929887] env[69702]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  684.929980] env[69702]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  684.930979] env[69702]: WARNING oslo_policy.policy [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  684.931185] env[69702]: WARNING oslo_policy.policy [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  684.932560] env[69702]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  684.947141] env[69702]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Session token - acquiring
[  684.947967] env[69702]: DEBUG networking_nsxv3.common.locking [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=69702) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.948197] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.948323] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.948705] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.958433] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.958679] env[69702]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  684.958908] env[69702]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69702) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  686.965188] env[69702]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Session token - acquiring
[  686.965419] env[69702]: DEBUG networking_nsxv3.common.locking [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=69702) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.965521] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.965707] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.966351] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.977201] env[69702]: DEBUG oslo_concurrency.lockutils [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69702) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.977201] env[69702]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  686.977201] env[69702]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69702) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  688.984224] env[69702]: CRITICAL neutron [None req-f3aa657f-63b9-48c8-8bd0-18b846f32d33 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  688.984224] env[69702]: ERROR neutron Traceback (most recent call last):
[  688.984224] env[69702]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  688.984224] env[69702]: ERROR neutron     sys.exit(main())
[  688.984224] env[69702]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  688.984224] env[69702]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  688.984224] env[69702]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  688.984224] env[69702]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  688.984224] env[69702]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  688.984224] env[69702]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  688.984224] env[69702]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  688.984224] env[69702]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  688.984224] env[69702]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  688.984224] env[69702]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  688.984224] env[69702]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  688.984224] env[69702]: ERROR neutron     response = self.get(path=path, params=params)
[  688.985422] env[69702]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  688.985422] env[69702]: ERROR neutron     raise RuntimeError(msg, last_err)
[  688.985422] env[69702]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  688.985422] env[69702]: ERROR neutron 
[  691.631482] env[70025]: INFO neutron.common.config [-] Logging enabled!
[  691.631746] env[70025]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  691.631989] env[70025]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70025) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  691.632238] env[70025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  691.632332] env[70025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  691.633181] env[70025]: WARNING oslo_policy.policy [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  691.633406] env[70025]: WARNING oslo_policy.policy [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  691.634643] env[70025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  691.648799] env[70025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Session token - acquiring
[  691.649597] env[70025]: DEBUG networking_nsxv3.common.locking [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70025) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.649799] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.649928] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.650298] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.660261] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.660519] env[70025]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  691.660668] env[70025]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70025) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  693.670325] env[70025]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Session token - acquiring
[  693.670325] env[70025]: DEBUG networking_nsxv3.common.locking [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70025) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.670325] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.670728] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.670728] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.678141] env[70025]: DEBUG oslo_concurrency.lockutils [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70025) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.678359] env[70025]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  693.678490] env[70025]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70025) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  695.680999] env[70025]: CRITICAL neutron [None req-60e97c2e-7f6c-4df7-a27b-5bf7caef54e9 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  695.680999] env[70025]: ERROR neutron Traceback (most recent call last):
[  695.680999] env[70025]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  695.680999] env[70025]: ERROR neutron     sys.exit(main())
[  695.680999] env[70025]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  695.680999] env[70025]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  695.680999] env[70025]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  695.680999] env[70025]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  695.680999] env[70025]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  695.680999] env[70025]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  695.680999] env[70025]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  695.680999] env[70025]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  695.680999] env[70025]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  695.680999] env[70025]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  695.680999] env[70025]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  695.680999] env[70025]: ERROR neutron     response = self.get(path=path, params=params)
[  695.681764] env[70025]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  695.681764] env[70025]: ERROR neutron     raise RuntimeError(msg, last_err)
[  695.681764] env[70025]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  695.681764] env[70025]: ERROR neutron 
[  698.371106] env[70214]: INFO neutron.common.config [-] Logging enabled!
[  698.371359] env[70214]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  698.371653] env[70214]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70214) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  698.371877] env[70214]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  698.371965] env[70214]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  698.372953] env[70214]: WARNING oslo_policy.policy [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  698.373048] env[70214]: WARNING oslo_policy.policy [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  698.374310] env[70214]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  698.391823] env[70214]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Session token - acquiring
[  698.392764] env[70214]: DEBUG networking_nsxv3.common.locking [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70214) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.392963] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.393089] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.393487] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.403196] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.403456] env[70214]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  698.403610] env[70214]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70214) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  700.409699] env[70214]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Session token - acquiring
[  700.410021] env[70214]: DEBUG networking_nsxv3.common.locking [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70214) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.410177] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.410323] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.410572] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.419924] env[70214]: DEBUG oslo_concurrency.lockutils [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70214) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.420184] env[70214]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  700.420313] env[70214]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70214) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  702.422106] env[70214]: CRITICAL neutron [None req-6133b016-259c-4bf7-a9e8-06c42233eb88 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  702.422106] env[70214]: ERROR neutron Traceback (most recent call last):
[  702.422106] env[70214]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  702.422106] env[70214]: ERROR neutron     sys.exit(main())
[  702.422106] env[70214]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  702.422106] env[70214]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  702.422106] env[70214]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  702.422106] env[70214]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  702.422106] env[70214]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  702.422106] env[70214]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  702.422106] env[70214]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  702.422106] env[70214]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  702.422106] env[70214]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  702.422106] env[70214]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  702.422106] env[70214]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  702.422106] env[70214]: ERROR neutron     response = self.get(path=path, params=params)
[  702.422865] env[70214]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  702.422865] env[70214]: ERROR neutron     raise RuntimeError(msg, last_err)
[  702.422865] env[70214]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  702.422865] env[70214]: ERROR neutron 
[  705.279710] env[70403]: INFO neutron.common.config [-] Logging enabled!
[  705.279967] env[70403]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  705.280300] env[70403]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70403) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  705.280493] env[70403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  705.280583] env[70403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  705.281421] env[70403]: WARNING oslo_policy.policy [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  705.281636] env[70403]: WARNING oslo_policy.policy [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  705.282857] env[70403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  705.297541] env[70403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Session token - acquiring
[  705.298340] env[70403]: DEBUG networking_nsxv3.common.locking [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70403) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.298533] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.298653] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.299007] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.308509] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.310440] env[70403]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  705.310694] env[70403]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70403) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  707.316885] env[70403]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Session token - acquiring
[  707.317221] env[70403]: DEBUG networking_nsxv3.common.locking [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70403) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.317362] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.317482] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.317762] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.327868] env[70403]: DEBUG oslo_concurrency.lockutils [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70403) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.328127] env[70403]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  707.328269] env[70403]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70403) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  709.331453] env[70403]: CRITICAL neutron [None req-5d9d8279-91da-4dfa-bda7-5a3e9b1e6d08 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  709.331453] env[70403]: ERROR neutron Traceback (most recent call last):
[  709.331453] env[70403]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  709.331453] env[70403]: ERROR neutron     sys.exit(main())
[  709.331453] env[70403]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  709.331453] env[70403]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  709.331453] env[70403]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  709.331453] env[70403]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  709.331453] env[70403]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  709.331453] env[70403]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  709.331453] env[70403]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  709.331453] env[70403]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  709.331453] env[70403]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  709.331453] env[70403]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  709.331453] env[70403]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  709.331453] env[70403]: ERROR neutron     response = self.get(path=path, params=params)
[  709.332399] env[70403]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  709.332399] env[70403]: ERROR neutron     raise RuntimeError(msg, last_err)
[  709.332399] env[70403]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  709.332399] env[70403]: ERROR neutron 
[  712.031697] env[70449]: INFO neutron.common.config [-] Logging enabled!
[  712.032110] env[70449]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  712.032652] env[70449]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70449) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  712.032788] env[70449]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  712.032883] env[70449]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  712.033768] env[70449]: WARNING oslo_policy.policy [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  712.034160] env[70449]: WARNING oslo_policy.policy [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  712.035639] env[70449]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  712.049868] env[70449]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Session token - acquiring
[  712.050749] env[70449]: DEBUG networking_nsxv3.common.locking [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70449) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.050961] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.051089] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.051467] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.061681] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.061936] env[70449]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  712.062091] env[70449]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70449) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  714.068318] env[70449]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Session token - acquiring
[  714.068709] env[70449]: DEBUG networking_nsxv3.common.locking [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70449) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.068842] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.068966] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.069242] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.078473] env[70449]: DEBUG oslo_concurrency.lockutils [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70449) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.078610] env[70449]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  714.078748] env[70449]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70449) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  716.080836] env[70449]: CRITICAL neutron [None req-bdbbf1a1-2509-436e-925b-90a5bbcdad33 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  716.080836] env[70449]: ERROR neutron Traceback (most recent call last):
[  716.080836] env[70449]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  716.080836] env[70449]: ERROR neutron     sys.exit(main())
[  716.080836] env[70449]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  716.080836] env[70449]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  716.080836] env[70449]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  716.080836] env[70449]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  716.080836] env[70449]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  716.080836] env[70449]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  716.080836] env[70449]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  716.080836] env[70449]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  716.080836] env[70449]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  716.080836] env[70449]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  716.080836] env[70449]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  716.080836] env[70449]: ERROR neutron     response = self.get(path=path, params=params)
[  716.081493] env[70449]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  716.081493] env[70449]: ERROR neutron     raise RuntimeError(msg, last_err)
[  716.081493] env[70449]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  716.081493] env[70449]: ERROR neutron 
[  718.968620] env[70491]: INFO neutron.common.config [-] Logging enabled!
[  718.968897] env[70491]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  718.969176] env[70491]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70491) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  718.969409] env[70491]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  718.969505] env[70491]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  718.970871] env[70491]: WARNING oslo_policy.policy [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  718.971108] env[70491]: WARNING oslo_policy.policy [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  718.973255] env[70491]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  718.995016] env[70491]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Session token - acquiring
[  718.995982] env[70491]: DEBUG networking_nsxv3.common.locking [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70491) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.996220] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.996347] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.996720] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.007096] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.007361] env[70491]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  719.007509] env[70491]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70491) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  721.013571] env[70491]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Session token - acquiring
[  721.014109] env[70491]: DEBUG networking_nsxv3.common.locking [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70491) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.014175] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.014319] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.014591] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.024301] env[70491]: DEBUG oslo_concurrency.lockutils [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70491) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.024579] env[70491]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  721.024758] env[70491]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70491) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  723.027678] env[70491]: CRITICAL neutron [None req-d3f70972-2a11-4773-9ff4-ce043559857c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  723.027678] env[70491]: ERROR neutron Traceback (most recent call last):
[  723.027678] env[70491]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  723.027678] env[70491]: ERROR neutron     sys.exit(main())
[  723.027678] env[70491]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  723.027678] env[70491]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  723.027678] env[70491]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  723.027678] env[70491]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  723.027678] env[70491]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  723.027678] env[70491]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  723.027678] env[70491]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  723.027678] env[70491]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  723.027678] env[70491]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  723.027678] env[70491]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  723.027678] env[70491]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  723.027678] env[70491]: ERROR neutron     response = self.get(path=path, params=params)
[  723.028495] env[70491]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  723.028495] env[70491]: ERROR neutron     raise RuntimeError(msg, last_err)
[  723.028495] env[70491]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  723.028495] env[70491]: ERROR neutron 
[  725.728611] env[70830]: INFO neutron.common.config [-] Logging enabled!
[  725.728962] env[70830]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  725.729125] env[70830]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70830) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  725.729342] env[70830]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  725.729455] env[70830]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  725.730278] env[70830]: WARNING oslo_policy.policy [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  725.730501] env[70830]: WARNING oslo_policy.policy [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  725.731679] env[70830]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  725.745921] env[70830]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Session token - acquiring
[  725.746659] env[70830]: DEBUG networking_nsxv3.common.locking [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70830) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  725.746855] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  725.746984] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  725.747339] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  725.756461] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  725.756734] env[70830]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  725.756844] env[70830]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70830) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  727.766840] env[70830]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Session token - acquiring
[  727.767199] env[70830]: DEBUG networking_nsxv3.common.locking [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70830) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  727.767338] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  727.767464] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  727.767798] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  727.777535] env[70830]: DEBUG oslo_concurrency.lockutils [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70830) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  727.777774] env[70830]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  727.777916] env[70830]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70830) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  729.780809] env[70830]: CRITICAL neutron [None req-e45e4856-69ca-47d9-8a39-0185fd5afbc1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  729.780809] env[70830]: ERROR neutron Traceback (most recent call last):
[  729.780809] env[70830]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  729.780809] env[70830]: ERROR neutron     sys.exit(main())
[  729.780809] env[70830]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  729.780809] env[70830]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  729.780809] env[70830]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  729.780809] env[70830]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  729.780809] env[70830]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  729.780809] env[70830]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  729.780809] env[70830]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  729.780809] env[70830]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  729.780809] env[70830]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  729.780809] env[70830]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  729.780809] env[70830]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  729.780809] env[70830]: ERROR neutron     response = self.get(path=path, params=params)
[  729.781546] env[70830]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  729.781546] env[70830]: ERROR neutron     raise RuntimeError(msg, last_err)
[  729.781546] env[70830]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  729.781546] env[70830]: ERROR neutron 
[  732.224098] env[70884]: INFO neutron.common.config [-] Logging enabled!
[  732.224490] env[70884]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  732.224677] env[70884]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70884) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  732.224960] env[70884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  732.225017] env[70884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  732.225913] env[70884]: WARNING oslo_policy.policy [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  732.226132] env[70884]: WARNING oslo_policy.policy [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  732.227361] env[70884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  732.242958] env[70884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Session token - acquiring
[  732.242958] env[70884]: DEBUG networking_nsxv3.common.locking [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70884) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  732.243185] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  732.243185] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  732.243575] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  732.252815] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  732.253074] env[70884]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  732.253224] env[70884]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70884) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  734.261857] env[70884]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Session token - acquiring
[  734.261857] env[70884]: DEBUG networking_nsxv3.common.locking [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70884) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  734.261857] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  734.262197] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  734.262197] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  734.270412] env[70884]: DEBUG oslo_concurrency.lockutils [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70884) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  734.270412] env[70884]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  734.270412] env[70884]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70884) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  736.272917] env[70884]: CRITICAL neutron [None req-41fad83c-5534-4e33-83b2-ec0348df114b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  736.272917] env[70884]: ERROR neutron Traceback (most recent call last):
[  736.272917] env[70884]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  736.272917] env[70884]: ERROR neutron     sys.exit(main())
[  736.272917] env[70884]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  736.272917] env[70884]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  736.272917] env[70884]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  736.272917] env[70884]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  736.272917] env[70884]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  736.272917] env[70884]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  736.272917] env[70884]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  736.272917] env[70884]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  736.272917] env[70884]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  736.272917] env[70884]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  736.272917] env[70884]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  736.272917] env[70884]: ERROR neutron     response = self.get(path=path, params=params)
[  736.273676] env[70884]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  736.273676] env[70884]: ERROR neutron     raise RuntimeError(msg, last_err)
[  736.273676] env[70884]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  736.273676] env[70884]: ERROR neutron 
[  739.209902] env[70989]: INFO neutron.common.config [-] Logging enabled!
[  739.210220] env[70989]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  739.210434] env[70989]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70989) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  739.210665] env[70989]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  739.210795] env[70989]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  739.211594] env[70989]: WARNING oslo_policy.policy [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  739.211811] env[70989]: WARNING oslo_policy.policy [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  739.213079] env[70989]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  739.227811] env[70989]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Session token - acquiring
[  739.228650] env[70989]: DEBUG networking_nsxv3.common.locking [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70989) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.228846] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.228969] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.229399] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.238816] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.239057] env[70989]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  739.239206] env[70989]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70989) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  741.245341] env[70989]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Session token - acquiring
[  741.245741] env[70989]: DEBUG networking_nsxv3.common.locking [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=70989) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.245951] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.246078] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.246349] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.264080] env[70989]: DEBUG oslo_concurrency.lockutils [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70989) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.264444] env[70989]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  741.264646] env[70989]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70989) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  743.266679] env[70989]: CRITICAL neutron [None req-5fc37764-788b-4461-badd-578f7d391af7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  743.266679] env[70989]: ERROR neutron Traceback (most recent call last):
[  743.266679] env[70989]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  743.266679] env[70989]: ERROR neutron     sys.exit(main())
[  743.266679] env[70989]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  743.266679] env[70989]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  743.266679] env[70989]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  743.266679] env[70989]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  743.266679] env[70989]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  743.266679] env[70989]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  743.266679] env[70989]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  743.266679] env[70989]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  743.266679] env[70989]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  743.266679] env[70989]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  743.266679] env[70989]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  743.266679] env[70989]: ERROR neutron     response = self.get(path=path, params=params)
[  743.267454] env[70989]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  743.267454] env[70989]: ERROR neutron     raise RuntimeError(msg, last_err)
[  743.267454] env[70989]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  743.267454] env[70989]: ERROR neutron 
[  746.151010] env[71157]: INFO neutron.common.config [-] Logging enabled!
[  746.151265] env[71157]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  746.151794] env[71157]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71157) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  746.151871] env[71157]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  746.151936] env[71157]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  746.152843] env[71157]: WARNING oslo_policy.policy [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  746.153092] env[71157]: WARNING oslo_policy.policy [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  746.154370] env[71157]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  746.170904] env[71157]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Session token - acquiring
[  746.176025] env[71157]: DEBUG networking_nsxv3.common.locking [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71157) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.176025] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.176385] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.176385] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.190192] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.190717] env[71157]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  746.190717] env[71157]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71157) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  748.196839] env[71157]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Session token - acquiring
[  748.197213] env[71157]: DEBUG networking_nsxv3.common.locking [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71157) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.197379] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.197509] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.197860] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.207293] env[71157]: DEBUG oslo_concurrency.lockutils [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71157) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.207537] env[71157]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  748.207670] env[71157]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71157) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  750.208929] env[71157]: CRITICAL neutron [None req-91e9508b-f65f-4813-acdf-4748196969ae None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  750.208929] env[71157]: ERROR neutron Traceback (most recent call last):
[  750.208929] env[71157]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  750.208929] env[71157]: ERROR neutron     sys.exit(main())
[  750.208929] env[71157]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  750.208929] env[71157]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  750.208929] env[71157]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  750.208929] env[71157]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  750.208929] env[71157]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  750.208929] env[71157]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  750.208929] env[71157]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  750.208929] env[71157]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  750.208929] env[71157]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  750.208929] env[71157]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  750.208929] env[71157]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  750.208929] env[71157]: ERROR neutron     response = self.get(path=path, params=params)
[  750.209933] env[71157]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  750.209933] env[71157]: ERROR neutron     raise RuntimeError(msg, last_err)
[  750.209933] env[71157]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  750.209933] env[71157]: ERROR neutron 
[  753.105871] env[71428]: INFO neutron.common.config [-] Logging enabled!
[  753.106221] env[71428]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  753.106423] env[71428]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71428) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  753.106681] env[71428]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  753.106781] env[71428]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  753.107705] env[71428]: WARNING oslo_policy.policy [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  753.107932] env[71428]: WARNING oslo_policy.policy [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  753.109370] env[71428]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  753.124016] env[71428]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Session token - acquiring
[  753.124911] env[71428]: DEBUG networking_nsxv3.common.locking [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71428) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.125128] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.125260] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.125645] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.134970] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.135231] env[71428]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  753.135384] env[71428]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71428) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  755.142093] env[71428]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Session token - acquiring
[  755.142485] env[71428]: DEBUG networking_nsxv3.common.locking [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71428) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.142698] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.142772] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.143025] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.152418] env[71428]: DEBUG oslo_concurrency.lockutils [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71428) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.152678] env[71428]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  755.152799] env[71428]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71428) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  757.155733] env[71428]: CRITICAL neutron [None req-60c7a219-e77a-43b6-9b52-ae0fd0e85059 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  757.155733] env[71428]: ERROR neutron Traceback (most recent call last):
[  757.155733] env[71428]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  757.155733] env[71428]: ERROR neutron     sys.exit(main())
[  757.155733] env[71428]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  757.155733] env[71428]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  757.155733] env[71428]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  757.155733] env[71428]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  757.155733] env[71428]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  757.155733] env[71428]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  757.155733] env[71428]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  757.155733] env[71428]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  757.155733] env[71428]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  757.155733] env[71428]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  757.155733] env[71428]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  757.155733] env[71428]: ERROR neutron     response = self.get(path=path, params=params)
[  757.156840] env[71428]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  757.156840] env[71428]: ERROR neutron     raise RuntimeError(msg, last_err)
[  757.156840] env[71428]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  757.156840] env[71428]: ERROR neutron 
[  759.758596] env[71513]: INFO neutron.common.config [-] Logging enabled!
[  759.758925] env[71513]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  759.759109] env[71513]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71513) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  759.759328] env[71513]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  759.759417] env[71513]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  759.760337] env[71513]: WARNING oslo_policy.policy [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  759.760543] env[71513]: WARNING oslo_policy.policy [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  759.761731] env[71513]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  759.775702] env[71513]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Session token - acquiring
[  759.776552] env[71513]: DEBUG networking_nsxv3.common.locking [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71513) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.776757] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.776880] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.777254] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.786976] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.787225] env[71513]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  759.787368] env[71513]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71513) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  761.795983] env[71513]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Session token - acquiring
[  761.795983] env[71513]: DEBUG networking_nsxv3.common.locking [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71513) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.795983] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.796450] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.796450] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.805876] env[71513]: DEBUG oslo_concurrency.lockutils [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71513) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.806339] env[71513]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  761.806988] env[71513]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71513) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  763.809193] env[71513]: CRITICAL neutron [None req-aadb14ab-ca86-4544-b642-1d4996aa9ead None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  763.809193] env[71513]: ERROR neutron Traceback (most recent call last):
[  763.809193] env[71513]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  763.809193] env[71513]: ERROR neutron     sys.exit(main())
[  763.809193] env[71513]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  763.809193] env[71513]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  763.809193] env[71513]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  763.809193] env[71513]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  763.809193] env[71513]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  763.809193] env[71513]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  763.809193] env[71513]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  763.809193] env[71513]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  763.809193] env[71513]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  763.809193] env[71513]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  763.809193] env[71513]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  763.809193] env[71513]: ERROR neutron     response = self.get(path=path, params=params)
[  763.809972] env[71513]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  763.809972] env[71513]: ERROR neutron     raise RuntimeError(msg, last_err)
[  763.809972] env[71513]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  763.809972] env[71513]: ERROR neutron 
[  766.526833] env[71608]: INFO neutron.common.config [-] Logging enabled!
[  766.527189] env[71608]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  766.527334] env[71608]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71608) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  766.527559] env[71608]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  766.527654] env[71608]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  766.528501] env[71608]: WARNING oslo_policy.policy [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  766.528731] env[71608]: WARNING oslo_policy.policy [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  766.529913] env[71608]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  766.544029] env[71608]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Session token - acquiring
[  766.544851] env[71608]: DEBUG networking_nsxv3.common.locking [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71608) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.545047] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.545176] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.545606] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.554202] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.554489] env[71608]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  766.554833] env[71608]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71608) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  768.560770] env[71608]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Session token - acquiring
[  768.561084] env[71608]: DEBUG networking_nsxv3.common.locking [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71608) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.561277] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.561403] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.561689] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.571036] env[71608]: DEBUG oslo_concurrency.lockutils [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71608) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.571294] env[71608]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  768.571448] env[71608]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71608) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  770.572912] env[71608]: CRITICAL neutron [None req-84fd83f5-9017-4758-8384-7d9835d8f2bb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  770.572912] env[71608]: ERROR neutron Traceback (most recent call last):
[  770.572912] env[71608]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  770.572912] env[71608]: ERROR neutron     sys.exit(main())
[  770.572912] env[71608]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  770.572912] env[71608]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  770.572912] env[71608]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  770.572912] env[71608]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  770.572912] env[71608]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  770.572912] env[71608]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  770.572912] env[71608]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  770.572912] env[71608]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  770.572912] env[71608]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  770.572912] env[71608]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  770.572912] env[71608]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  770.572912] env[71608]: ERROR neutron     response = self.get(path=path, params=params)
[  770.573648] env[71608]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  770.573648] env[71608]: ERROR neutron     raise RuntimeError(msg, last_err)
[  770.573648] env[71608]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  770.573648] env[71608]: ERROR neutron 
[  773.297481] env[71723]: INFO neutron.common.config [-] Logging enabled!
[  773.297796] env[71723]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  773.298020] env[71723]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71723) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  773.298246] env[71723]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  773.298337] env[71723]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  773.299199] env[71723]: WARNING oslo_policy.policy [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  773.299412] env[71723]: WARNING oslo_policy.policy [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  773.300715] env[71723]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  773.315221] env[71723]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Session token - acquiring
[  773.316070] env[71723]: DEBUG networking_nsxv3.common.locking [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71723) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  773.316300] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  773.316459] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  773.316886] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  773.326314] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  773.326592] env[71723]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  773.326776] env[71723]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71723) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  775.332976] env[71723]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Session token - acquiring
[  775.333364] env[71723]: DEBUG networking_nsxv3.common.locking [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71723) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  775.333563] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  775.333683] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  775.333961] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  775.343058] env[71723]: DEBUG oslo_concurrency.lockutils [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71723) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.343372] env[71723]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  775.343527] env[71723]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71723) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  777.344904] env[71723]: CRITICAL neutron [None req-9acd0d3a-28fb-4d8b-a9fc-3c085a24a1a1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  777.344904] env[71723]: ERROR neutron Traceback (most recent call last):
[  777.344904] env[71723]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  777.344904] env[71723]: ERROR neutron     sys.exit(main())
[  777.344904] env[71723]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  777.344904] env[71723]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  777.344904] env[71723]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  777.344904] env[71723]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  777.344904] env[71723]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  777.344904] env[71723]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  777.344904] env[71723]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  777.344904] env[71723]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  777.344904] env[71723]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  777.344904] env[71723]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  777.344904] env[71723]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  777.344904] env[71723]: ERROR neutron     response = self.get(path=path, params=params)
[  777.345732] env[71723]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  777.345732] env[71723]: ERROR neutron     raise RuntimeError(msg, last_err)
[  777.345732] env[71723]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  777.345732] env[71723]: ERROR neutron 
[  780.165773] env[71831]: INFO neutron.common.config [-] Logging enabled!
[  780.166071] env[71831]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  780.166392] env[71831]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71831) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  780.166611] env[71831]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  780.166712] env[71831]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  780.167576] env[71831]: WARNING oslo_policy.policy [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  780.167801] env[71831]: WARNING oslo_policy.policy [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  780.169549] env[71831]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  780.185160] env[71831]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Session token - acquiring
[  780.186266] env[71831]: DEBUG networking_nsxv3.common.locking [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71831) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  780.186557] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  780.186715] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  780.187307] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  780.197343] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  780.197666] env[71831]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  780.197853] env[71831]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71831) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  782.203899] env[71831]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Session token - acquiring
[  782.204245] env[71831]: DEBUG networking_nsxv3.common.locking [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71831) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  782.204496] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  782.204633] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  782.204907] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  782.214756] env[71831]: DEBUG oslo_concurrency.lockutils [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71831) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  782.215375] env[71831]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  782.215375] env[71831]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71831) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  784.216886] env[71831]: CRITICAL neutron [None req-f99ebf52-17ec-4c0a-823a-e3c65d7f03d7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  784.216886] env[71831]: ERROR neutron Traceback (most recent call last):
[  784.216886] env[71831]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  784.216886] env[71831]: ERROR neutron     sys.exit(main())
[  784.216886] env[71831]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  784.216886] env[71831]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  784.216886] env[71831]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  784.216886] env[71831]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  784.216886] env[71831]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  784.216886] env[71831]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  784.216886] env[71831]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  784.216886] env[71831]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  784.216886] env[71831]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  784.216886] env[71831]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  784.216886] env[71831]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  784.216886] env[71831]: ERROR neutron     response = self.get(path=path, params=params)
[  784.217590] env[71831]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  784.217590] env[71831]: ERROR neutron     raise RuntimeError(msg, last_err)
[  784.217590] env[71831]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  784.217590] env[71831]: ERROR neutron 
[  786.803790] env[71986]: INFO neutron.common.config [-] Logging enabled!
[  786.804229] env[71986]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  786.804435] env[71986]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71986) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  786.804686] env[71986]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  786.804780] env[71986]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  786.805661] env[71986]: WARNING oslo_policy.policy [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  786.805897] env[71986]: WARNING oslo_policy.policy [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  786.807165] env[71986]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  786.821599] env[71986]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Session token - acquiring
[  786.822425] env[71986]: DEBUG networking_nsxv3.common.locking [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71986) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.822543] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.823160] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.823160] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.832166] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.832390] env[71986]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  786.832542] env[71986]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71986) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  788.838778] env[71986]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Session token - acquiring
[  788.839137] env[71986]: DEBUG networking_nsxv3.common.locking [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=71986) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.839330] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.839457] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.839735] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.849169] env[71986]: DEBUG oslo_concurrency.lockutils [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71986) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.849418] env[71986]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  788.849559] env[71986]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71986) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  790.852479] env[71986]: CRITICAL neutron [None req-ca8daf77-cb63-4c38-91b8-3ce9c7ff3970 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  790.852479] env[71986]: ERROR neutron Traceback (most recent call last):
[  790.852479] env[71986]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  790.852479] env[71986]: ERROR neutron     sys.exit(main())
[  790.852479] env[71986]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  790.852479] env[71986]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  790.852479] env[71986]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  790.852479] env[71986]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  790.852479] env[71986]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  790.852479] env[71986]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  790.852479] env[71986]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  790.852479] env[71986]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  790.852479] env[71986]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  790.852479] env[71986]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  790.852479] env[71986]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  790.852479] env[71986]: ERROR neutron     response = self.get(path=path, params=params)
[  790.853258] env[71986]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  790.853258] env[71986]: ERROR neutron     raise RuntimeError(msg, last_err)
[  790.853258] env[71986]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  790.853258] env[71986]: ERROR neutron 
[  793.556451] env[72106]: INFO neutron.common.config [-] Logging enabled!
[  793.556863] env[72106]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  793.556953] env[72106]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72106) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  793.557173] env[72106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  793.557268] env[72106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  793.558085] env[72106]: WARNING oslo_policy.policy [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  793.558296] env[72106]: WARNING oslo_policy.policy [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  793.559457] env[72106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  793.573617] env[72106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Session token - acquiring
[  793.574354] env[72106]: DEBUG networking_nsxv3.common.locking [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72106) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.574545] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.574674] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.575018] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.583752] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.583987] env[72106]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  793.584149] env[72106]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72106) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  795.592230] env[72106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Session token - acquiring
[  795.592635] env[72106]: DEBUG networking_nsxv3.common.locking [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72106) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  795.592854] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  795.592978] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  795.593247] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  795.602436] env[72106]: DEBUG oslo_concurrency.lockutils [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72106) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  795.602686] env[72106]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  795.602822] env[72106]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72106) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  797.605085] env[72106]: CRITICAL neutron [None req-43927650-e42e-4945-85c8-e1a79e1dded1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  797.605085] env[72106]: ERROR neutron Traceback (most recent call last):
[  797.605085] env[72106]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  797.605085] env[72106]: ERROR neutron     sys.exit(main())
[  797.605085] env[72106]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  797.605085] env[72106]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  797.605085] env[72106]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  797.605085] env[72106]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  797.605085] env[72106]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  797.605085] env[72106]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  797.605085] env[72106]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  797.605085] env[72106]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  797.605085] env[72106]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  797.605085] env[72106]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  797.605085] env[72106]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  797.605085] env[72106]: ERROR neutron     response = self.get(path=path, params=params)
[  797.606026] env[72106]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  797.606026] env[72106]: ERROR neutron     raise RuntimeError(msg, last_err)
[  797.606026] env[72106]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  797.606026] env[72106]: ERROR neutron 
[  800.392045] env[72265]: INFO neutron.common.config [-] Logging enabled!
[  800.392406] env[72265]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  800.392586] env[72265]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72265) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  800.392819] env[72265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  800.392914] env[72265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  800.393752] env[72265]: WARNING oslo_policy.policy [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  800.393980] env[72265]: WARNING oslo_policy.policy [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  800.395223] env[72265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  800.409632] env[72265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Session token - acquiring
[  800.410415] env[72265]: DEBUG networking_nsxv3.common.locking [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72265) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  800.410615] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  800.410740] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  800.411101] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  800.420354] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  800.420616] env[72265]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  800.420770] env[72265]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72265) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  802.427040] env[72265]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Session token - acquiring
[  802.427341] env[72265]: DEBUG networking_nsxv3.common.locking [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72265) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.427645] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.427645] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.427914] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.437628] env[72265]: DEBUG oslo_concurrency.lockutils [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72265) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.438121] env[72265]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  802.438363] env[72265]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72265) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  804.440948] env[72265]: CRITICAL neutron [None req-30d1f577-5d38-4b81-b972-b5ccd1c6dc74 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  804.440948] env[72265]: ERROR neutron Traceback (most recent call last):
[  804.440948] env[72265]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  804.440948] env[72265]: ERROR neutron     sys.exit(main())
[  804.440948] env[72265]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  804.440948] env[72265]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  804.440948] env[72265]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  804.440948] env[72265]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  804.440948] env[72265]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  804.440948] env[72265]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  804.440948] env[72265]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  804.440948] env[72265]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  804.440948] env[72265]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  804.440948] env[72265]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  804.440948] env[72265]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  804.440948] env[72265]: ERROR neutron     response = self.get(path=path, params=params)
[  804.441919] env[72265]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  804.441919] env[72265]: ERROR neutron     raise RuntimeError(msg, last_err)
[  804.441919] env[72265]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  804.441919] env[72265]: ERROR neutron 
[  807.029971] env[72390]: INFO neutron.common.config [-] Logging enabled!
[  807.030326] env[72390]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  807.030492] env[72390]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72390) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  807.030721] env[72390]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  807.030815] env[72390]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  807.031729] env[72390]: WARNING oslo_policy.policy [None req-379e1449-8929-4967-9e76-87c3be317864 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  807.031887] env[72390]: WARNING oslo_policy.policy [None req-379e1449-8929-4967-9e76-87c3be317864 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  807.033143] env[72390]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  807.047628] env[72390]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Session token - acquiring
[  807.048457] env[72390]: DEBUG networking_nsxv3.common.locking [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72390) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.048658] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.048784] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.049165] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.058251] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.058492] env[72390]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  807.058645] env[72390]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72390) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  809.064852] env[72390]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Session token - acquiring
[  809.065204] env[72390]: DEBUG networking_nsxv3.common.locking [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72390) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.065363] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.065487] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.065759] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.076803] env[72390]: DEBUG oslo_concurrency.lockutils [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72390) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.077052] env[72390]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  809.077190] env[72390]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72390) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  811.080633] env[72390]: CRITICAL neutron [None req-379e1449-8929-4967-9e76-87c3be317864 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  811.080633] env[72390]: ERROR neutron Traceback (most recent call last):
[  811.080633] env[72390]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  811.080633] env[72390]: ERROR neutron     sys.exit(main())
[  811.080633] env[72390]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  811.080633] env[72390]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  811.080633] env[72390]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  811.080633] env[72390]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  811.080633] env[72390]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  811.080633] env[72390]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  811.080633] env[72390]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  811.080633] env[72390]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  811.080633] env[72390]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  811.080633] env[72390]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  811.080633] env[72390]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  811.080633] env[72390]: ERROR neutron     response = self.get(path=path, params=params)
[  811.081379] env[72390]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  811.081379] env[72390]: ERROR neutron     raise RuntimeError(msg, last_err)
[  811.081379] env[72390]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  811.081379] env[72390]: ERROR neutron 
[  813.778925] env[72621]: INFO neutron.common.config [-] Logging enabled!
[  813.779304] env[72621]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  813.779466] env[72621]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72621) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  813.779695] env[72621]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  813.779790] env[72621]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  813.780645] env[72621]: WARNING oslo_policy.policy [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  813.780873] env[72621]: WARNING oslo_policy.policy [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  813.782091] env[72621]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  813.797677] env[72621]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Session token - acquiring
[  813.798560] env[72621]: DEBUG networking_nsxv3.common.locking [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72621) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.798778] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.798906] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.799288] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.808572] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.808849] env[72621]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  813.809004] env[72621]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72621) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  815.815150] env[72621]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Session token - acquiring
[  815.815470] env[72621]: DEBUG networking_nsxv3.common.locking [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72621) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.815666] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.816536] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.816536] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.827008] env[72621]: DEBUG oslo_concurrency.lockutils [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72621) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.827008] env[72621]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  815.827008] env[72621]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72621) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  817.829170] env[72621]: CRITICAL neutron [None req-192ad706-ba10-417e-ad25-6a2117fa8a4c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  817.829170] env[72621]: ERROR neutron Traceback (most recent call last):
[  817.829170] env[72621]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  817.829170] env[72621]: ERROR neutron     sys.exit(main())
[  817.829170] env[72621]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  817.829170] env[72621]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  817.829170] env[72621]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  817.829170] env[72621]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  817.829170] env[72621]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  817.829170] env[72621]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  817.829170] env[72621]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  817.829170] env[72621]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  817.829170] env[72621]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  817.829170] env[72621]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  817.829170] env[72621]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  817.829170] env[72621]: ERROR neutron     response = self.get(path=path, params=params)
[  817.830029] env[72621]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  817.830029] env[72621]: ERROR neutron     raise RuntimeError(msg, last_err)
[  817.830029] env[72621]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  817.830029] env[72621]: ERROR neutron 
[  820.552932] env[72744]: INFO neutron.common.config [-] Logging enabled!
[  820.553325] env[72744]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  820.553520] env[72744]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72744) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  820.553792] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  820.553888] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  820.554932] env[72744]: WARNING oslo_policy.policy [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  820.555301] env[72744]: WARNING oslo_policy.policy [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  820.556914] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  820.573084] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Session token - acquiring
[  820.573898] env[72744]: DEBUG networking_nsxv3.common.locking [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72744) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  820.574105] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  820.574232] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  820.574627] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  820.583584] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  820.583840] env[72744]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  820.583989] env[72744]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72744) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  822.589815] env[72744]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Session token - acquiring
[  822.590181] env[72744]: DEBUG networking_nsxv3.common.locking [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72744) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.590290] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.590415] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.590690] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.599571] env[72744]: DEBUG oslo_concurrency.lockutils [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72744) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.599805] env[72744]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  822.599940] env[72744]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72744) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  824.601632] env[72744]: CRITICAL neutron [None req-a0242456-3156-4d7d-a944-f85367f8b43c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  824.601632] env[72744]: ERROR neutron Traceback (most recent call last):
[  824.601632] env[72744]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  824.601632] env[72744]: ERROR neutron     sys.exit(main())
[  824.601632] env[72744]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  824.601632] env[72744]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  824.601632] env[72744]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  824.601632] env[72744]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  824.601632] env[72744]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  824.601632] env[72744]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  824.601632] env[72744]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  824.601632] env[72744]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  824.601632] env[72744]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  824.601632] env[72744]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  824.601632] env[72744]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  824.601632] env[72744]: ERROR neutron     response = self.get(path=path, params=params)
[  824.602502] env[72744]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  824.602502] env[72744]: ERROR neutron     raise RuntimeError(msg, last_err)
[  824.602502] env[72744]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  824.602502] env[72744]: ERROR neutron 
[  827.351641] env[72877]: INFO neutron.common.config [-] Logging enabled!
[  827.351936] env[72877]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  827.352176] env[72877]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72877) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  827.352441] env[72877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  827.352652] env[72877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  827.353375] env[72877]: WARNING oslo_policy.policy [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  827.353606] env[72877]: WARNING oslo_policy.policy [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  827.354851] env[72877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  827.385564] env[72877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Session token - acquiring
[  827.386438] env[72877]: DEBUG networking_nsxv3.common.locking [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72877) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.386662] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.386791] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.387203] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.398468] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.398750] env[72877]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  827.398926] env[72877]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72877) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  829.405052] env[72877]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Session token - acquiring
[  829.405386] env[72877]: DEBUG networking_nsxv3.common.locking [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=72877) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.405658] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.405792] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.406057] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.415108] env[72877]: DEBUG oslo_concurrency.lockutils [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72877) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.415352] env[72877]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  829.415513] env[72877]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72877) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  831.418083] env[72877]: CRITICAL neutron [None req-0cdcd59b-0d23-47cb-b04c-fcb46d2503d5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  831.418083] env[72877]: ERROR neutron Traceback (most recent call last):
[  831.418083] env[72877]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  831.418083] env[72877]: ERROR neutron     sys.exit(main())
[  831.418083] env[72877]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  831.418083] env[72877]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  831.418083] env[72877]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  831.418083] env[72877]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  831.418083] env[72877]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  831.418083] env[72877]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  831.418083] env[72877]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  831.418083] env[72877]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  831.418083] env[72877]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  831.418083] env[72877]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  831.418083] env[72877]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  831.418083] env[72877]: ERROR neutron     response = self.get(path=path, params=params)
[  831.418834] env[72877]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  831.418834] env[72877]: ERROR neutron     raise RuntimeError(msg, last_err)
[  831.418834] env[72877]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  831.418834] env[72877]: ERROR neutron 
[  834.072599] env[73097]: INFO neutron.common.config [-] Logging enabled!
[  834.072937] env[73097]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  834.073112] env[73097]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73097) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  834.073341] env[73097]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  834.073435] env[73097]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  834.074286] env[73097]: WARNING oslo_policy.policy [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  834.074520] env[73097]: WARNING oslo_policy.policy [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  834.075763] env[73097]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  834.090213] env[73097]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Session token - acquiring
[  834.090857] env[73097]: DEBUG networking_nsxv3.common.locking [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73097) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.091066] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.091193] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.091552] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.100723] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.101125] env[73097]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  834.101342] env[73097]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73097) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  836.107780] env[73097]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Session token - acquiring
[  836.108156] env[73097]: DEBUG networking_nsxv3.common.locking [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73097) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.108681] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.108681] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.108681] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.118627] env[73097]: DEBUG oslo_concurrency.lockutils [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73097) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.118912] env[73097]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  836.119099] env[73097]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73097) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  838.120890] env[73097]: CRITICAL neutron [None req-892025c9-327e-42ca-ac4a-b0967992a02c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  838.120890] env[73097]: ERROR neutron Traceback (most recent call last):
[  838.120890] env[73097]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  838.120890] env[73097]: ERROR neutron     sys.exit(main())
[  838.120890] env[73097]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  838.120890] env[73097]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  838.120890] env[73097]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  838.120890] env[73097]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  838.120890] env[73097]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  838.120890] env[73097]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  838.120890] env[73097]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  838.120890] env[73097]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  838.120890] env[73097]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  838.120890] env[73097]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  838.120890] env[73097]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  838.120890] env[73097]: ERROR neutron     response = self.get(path=path, params=params)
[  838.121573] env[73097]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  838.121573] env[73097]: ERROR neutron     raise RuntimeError(msg, last_err)
[  838.121573] env[73097]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  838.121573] env[73097]: ERROR neutron 
[  840.730722] env[73344]: INFO neutron.common.config [-] Logging enabled!
[  840.731121] env[73344]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  840.731324] env[73344]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73344) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  840.731499] env[73344]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  840.731591] env[73344]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  840.732441] env[73344]: WARNING oslo_policy.policy [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  840.732730] env[73344]: WARNING oslo_policy.policy [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  840.733882] env[73344]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  840.748500] env[73344]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Session token - acquiring
[  840.749414] env[73344]: DEBUG networking_nsxv3.common.locking [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73344) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.749694] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.749851] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.750263] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.759858] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.760178] env[73344]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  840.760376] env[73344]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73344) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  842.766309] env[73344]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Session token - acquiring
[  842.766615] env[73344]: DEBUG networking_nsxv3.common.locking [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73344) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.766797] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.766919] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.767199] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.776523] env[73344]: DEBUG oslo_concurrency.lockutils [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73344) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.776760] env[73344]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  842.776914] env[73344]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73344) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  844.779801] env[73344]: CRITICAL neutron [None req-737ed68b-d4b7-431d-a92c-075bb6a4bd37 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  844.779801] env[73344]: ERROR neutron Traceback (most recent call last):
[  844.779801] env[73344]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  844.779801] env[73344]: ERROR neutron     sys.exit(main())
[  844.779801] env[73344]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  844.779801] env[73344]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  844.779801] env[73344]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  844.779801] env[73344]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  844.779801] env[73344]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  844.779801] env[73344]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  844.779801] env[73344]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  844.779801] env[73344]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  844.779801] env[73344]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  844.779801] env[73344]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  844.779801] env[73344]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  844.779801] env[73344]: ERROR neutron     response = self.get(path=path, params=params)
[  844.780610] env[73344]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  844.780610] env[73344]: ERROR neutron     raise RuntimeError(msg, last_err)
[  844.780610] env[73344]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  844.780610] env[73344]: ERROR neutron 
[  847.569775] env[73473]: INFO neutron.common.config [-] Logging enabled!
[  847.570141] env[73473]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  847.570326] env[73473]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73473) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  847.570567] env[73473]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  847.570653] env[73473]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  847.571543] env[73473]: WARNING oslo_policy.policy [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  847.571785] env[73473]: WARNING oslo_policy.policy [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  847.573085] env[73473]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  847.590916] env[73473]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Session token - acquiring
[  847.591791] env[73473]: DEBUG networking_nsxv3.common.locking [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73473) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.592019] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.592152] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.592610] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.603435] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.603435] env[73473]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  847.604226] env[73473]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73473) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  849.609797] env[73473]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Session token - acquiring
[  849.610098] env[73473]: DEBUG networking_nsxv3.common.locking [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73473) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.610309] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.610385] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.610653] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.619731] env[73473]: DEBUG oslo_concurrency.lockutils [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73473) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.619976] env[73473]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  849.620131] env[73473]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73473) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  851.623018] env[73473]: CRITICAL neutron [None req-514150a7-ff15-4bcc-899c-b76b4065d922 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  851.623018] env[73473]: ERROR neutron Traceback (most recent call last):
[  851.623018] env[73473]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  851.623018] env[73473]: ERROR neutron     sys.exit(main())
[  851.623018] env[73473]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  851.623018] env[73473]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  851.623018] env[73473]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  851.623018] env[73473]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  851.623018] env[73473]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  851.623018] env[73473]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  851.623018] env[73473]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  851.623018] env[73473]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  851.623018] env[73473]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  851.623018] env[73473]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  851.623018] env[73473]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  851.623018] env[73473]: ERROR neutron     response = self.get(path=path, params=params)
[  851.623748] env[73473]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  851.623748] env[73473]: ERROR neutron     raise RuntimeError(msg, last_err)
[  851.623748] env[73473]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  851.623748] env[73473]: ERROR neutron 
[  854.206642] env[73692]: INFO neutron.common.config [-] Logging enabled!
[  854.206927] env[73692]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  854.207114] env[73692]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73692) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  854.207320] env[73692]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  854.207427] env[73692]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  854.208520] env[73692]: WARNING oslo_policy.policy [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  854.208520] env[73692]: WARNING oslo_policy.policy [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  854.209817] env[73692]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  854.230277] env[73692]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Session token - acquiring
[  854.231165] env[73692]: DEBUG networking_nsxv3.common.locking [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73692) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.231376] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.231503] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.231895] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.241087] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.241326] env[73692]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  854.241495] env[73692]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73692) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  856.247453] env[73692]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Session token - acquiring
[  856.247774] env[73692]: DEBUG networking_nsxv3.common.locking [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73692) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.247940] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.248077] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.248352] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.257390] env[73692]: DEBUG oslo_concurrency.lockutils [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73692) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.257646] env[73692]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  856.257787] env[73692]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73692) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  858.281448] env[73692]: CRITICAL neutron [None req-caaa329f-ccb1-4bbc-9f1d-3754424189c7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  858.281448] env[73692]: ERROR neutron Traceback (most recent call last):
[  858.281448] env[73692]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  858.281448] env[73692]: ERROR neutron     sys.exit(main())
[  858.281448] env[73692]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  858.281448] env[73692]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  858.281448] env[73692]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  858.281448] env[73692]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  858.281448] env[73692]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  858.281448] env[73692]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  858.281448] env[73692]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  858.281448] env[73692]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  858.281448] env[73692]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  858.281448] env[73692]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  858.281448] env[73692]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  858.281448] env[73692]: ERROR neutron     response = self.get(path=path, params=params)
[  858.282152] env[73692]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  858.282152] env[73692]: ERROR neutron     raise RuntimeError(msg, last_err)
[  858.282152] env[73692]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  858.282152] env[73692]: ERROR neutron 
[  860.952312] env[73866]: INFO neutron.common.config [-] Logging enabled!
[  860.952646] env[73866]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  860.952883] env[73866]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73866) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  860.953191] env[73866]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  860.953245] env[73866]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  860.954051] env[73866]: WARNING oslo_policy.policy [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  860.954276] env[73866]: WARNING oslo_policy.policy [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  860.955554] env[73866]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  860.971772] env[73866]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Session token - acquiring
[  860.972862] env[73866]: DEBUG networking_nsxv3.common.locking [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73866) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.973081] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.973211] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.973586] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.983028] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.983289] env[73866]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  860.983441] env[73866]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73866) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  862.990261] env[73866]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Session token - acquiring
[  862.990620] env[73866]: DEBUG networking_nsxv3.common.locking [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73866) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  862.990843] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  862.994536] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  862.994536] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.005526] env[73866]: DEBUG oslo_concurrency.lockutils [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73866) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.005787] env[73866]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  863.006042] env[73866]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73866) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  865.008893] env[73866]: CRITICAL neutron [None req-1757575b-f0f2-478f-8c14-8ca5a7df5384 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  865.008893] env[73866]: ERROR neutron Traceback (most recent call last):
[  865.008893] env[73866]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  865.008893] env[73866]: ERROR neutron     sys.exit(main())
[  865.008893] env[73866]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  865.008893] env[73866]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  865.008893] env[73866]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  865.008893] env[73866]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  865.008893] env[73866]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  865.008893] env[73866]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  865.008893] env[73866]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  865.008893] env[73866]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  865.008893] env[73866]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  865.008893] env[73866]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  865.008893] env[73866]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  865.008893] env[73866]: ERROR neutron     response = self.get(path=path, params=params)
[  865.009493] env[73866]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  865.009493] env[73866]: ERROR neutron     raise RuntimeError(msg, last_err)
[  865.009493] env[73866]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  865.009493] env[73866]: ERROR neutron 
[  867.729205] env[73993]: INFO neutron.common.config [-] Logging enabled!
[  867.729565] env[73993]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  867.729723] env[73993]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73993) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  867.729965] env[73993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  867.730053] env[73993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  867.730891] env[73993]: WARNING oslo_policy.policy [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  867.731102] env[73993]: WARNING oslo_policy.policy [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  867.732341] env[73993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  867.746534] env[73993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Session token - acquiring
[  867.747332] env[73993]: DEBUG networking_nsxv3.common.locking [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73993) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  867.747529] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  867.747650] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  867.748028] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  867.756916] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  867.757165] env[73993]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  867.757322] env[73993]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73993) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  869.763475] env[73993]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Session token - acquiring
[  869.763957] env[73993]: DEBUG networking_nsxv3.common.locking [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=73993) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  869.764176] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  869.764299] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  869.764598] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  869.773472] env[73993]: DEBUG oslo_concurrency.lockutils [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73993) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  869.773714] env[73993]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  869.773857] env[73993]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73993) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  871.776779] env[73993]: CRITICAL neutron [None req-2eab2155-54cc-42f2-86e6-6a92627c18b0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  871.776779] env[73993]: ERROR neutron Traceback (most recent call last):
[  871.776779] env[73993]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  871.776779] env[73993]: ERROR neutron     sys.exit(main())
[  871.776779] env[73993]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  871.776779] env[73993]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  871.776779] env[73993]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  871.776779] env[73993]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  871.776779] env[73993]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  871.776779] env[73993]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  871.776779] env[73993]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  871.776779] env[73993]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  871.776779] env[73993]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  871.776779] env[73993]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  871.776779] env[73993]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  871.776779] env[73993]: ERROR neutron     response = self.get(path=path, params=params)
[  871.777332] env[73993]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  871.777332] env[73993]: ERROR neutron     raise RuntimeError(msg, last_err)
[  871.777332] env[73993]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  871.777332] env[73993]: ERROR neutron 
[  874.275339] env[74225]: INFO neutron.common.config [-] Logging enabled!
[  874.275808] env[74225]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  874.276913] env[74225]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74225) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  874.277151] env[74225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  874.277244] env[74225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  874.278117] env[74225]: WARNING oslo_policy.policy [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  874.278346] env[74225]: WARNING oslo_policy.policy [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  874.279580] env[74225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  874.293976] env[74225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Session token - acquiring
[  874.294822] env[74225]: DEBUG networking_nsxv3.common.locking [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74225) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  874.295023] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  874.295155] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  874.295537] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  874.304903] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  874.305116] env[74225]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  874.305264] env[74225]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74225) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  876.311379] env[74225]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Session token - acquiring
[  876.311758] env[74225]: DEBUG networking_nsxv3.common.locking [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74225) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.311999] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.312147] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.312425] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.321603] env[74225]: DEBUG oslo_concurrency.lockutils [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74225) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.321832] env[74225]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  876.321972] env[74225]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74225) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  878.324876] env[74225]: CRITICAL neutron [None req-61a8d620-6218-4cc5-bec3-726d7fbb3dc6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  878.324876] env[74225]: ERROR neutron Traceback (most recent call last):
[  878.324876] env[74225]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  878.324876] env[74225]: ERROR neutron     sys.exit(main())
[  878.324876] env[74225]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  878.324876] env[74225]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  878.324876] env[74225]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  878.324876] env[74225]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  878.324876] env[74225]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  878.324876] env[74225]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  878.324876] env[74225]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  878.324876] env[74225]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  878.324876] env[74225]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  878.324876] env[74225]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  878.324876] env[74225]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  878.324876] env[74225]: ERROR neutron     response = self.get(path=path, params=params)
[  878.325691] env[74225]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  878.325691] env[74225]: ERROR neutron     raise RuntimeError(msg, last_err)
[  878.325691] env[74225]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  878.325691] env[74225]: ERROR neutron 
[  881.095779] env[74360]: INFO neutron.common.config [-] Logging enabled!
[  881.096146] env[74360]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  881.096389] env[74360]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74360) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  881.097288] env[74360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  881.097288] env[74360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  881.097620] env[74360]: WARNING oslo_policy.policy [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  881.097760] env[74360]: WARNING oslo_policy.policy [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  881.099074] env[74360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  881.113206] env[74360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Session token - acquiring
[  881.114121] env[74360]: DEBUG networking_nsxv3.common.locking [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74360) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.114296] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.114447] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.114835] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.130636] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.130916] env[74360]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  881.131062] env[74360]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74360) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  883.137119] env[74360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Session token - acquiring
[  883.137480] env[74360]: DEBUG networking_nsxv3.common.locking [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74360) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  883.137648] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  883.137778] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  883.138040] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  883.158118] env[74360]: DEBUG oslo_concurrency.lockutils [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74360) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  883.158620] env[74360]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  883.158797] env[74360]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74360) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  885.160856] env[74360]: CRITICAL neutron [None req-ebfc77a7-419b-4437-bd41-14e051ce18a8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  885.160856] env[74360]: ERROR neutron Traceback (most recent call last):
[  885.160856] env[74360]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  885.160856] env[74360]: ERROR neutron     sys.exit(main())
[  885.160856] env[74360]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  885.160856] env[74360]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  885.160856] env[74360]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  885.160856] env[74360]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  885.160856] env[74360]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  885.160856] env[74360]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  885.160856] env[74360]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  885.160856] env[74360]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  885.160856] env[74360]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  885.160856] env[74360]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  885.160856] env[74360]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  885.160856] env[74360]: ERROR neutron     response = self.get(path=path, params=params)
[  885.161646] env[74360]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  885.161646] env[74360]: ERROR neutron     raise RuntimeError(msg, last_err)
[  885.161646] env[74360]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  885.161646] env[74360]: ERROR neutron 
[  887.826787] env[74598]: INFO neutron.common.config [-] Logging enabled!
[  887.826787] env[74598]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  887.827031] env[74598]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74598) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  887.828039] env[74598]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  887.828039] env[74598]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  887.828845] env[74598]: WARNING oslo_policy.policy [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  887.828845] env[74598]: WARNING oslo_policy.policy [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  887.829596] env[74598]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  887.850667] env[74598]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Session token - acquiring
[  887.851536] env[74598]: DEBUG networking_nsxv3.common.locking [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74598) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.851748] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.851876] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.852288] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.861453] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.861729] env[74598]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  887.861879] env[74598]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74598) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  889.868451] env[74598]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Session token - acquiring
[  889.868451] env[74598]: DEBUG networking_nsxv3.common.locking [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74598) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.868797] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.868797] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.869067] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.877881] env[74598]: DEBUG oslo_concurrency.lockutils [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74598) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.878119] env[74598]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  889.878257] env[74598]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74598) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  891.880883] env[74598]: CRITICAL neutron [None req-d7e3ea9c-ac33-4671-90e0-7c8354299088 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  891.880883] env[74598]: ERROR neutron Traceback (most recent call last):
[  891.880883] env[74598]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  891.880883] env[74598]: ERROR neutron     sys.exit(main())
[  891.880883] env[74598]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  891.880883] env[74598]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  891.880883] env[74598]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  891.880883] env[74598]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  891.880883] env[74598]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  891.880883] env[74598]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  891.880883] env[74598]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  891.880883] env[74598]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  891.880883] env[74598]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  891.880883] env[74598]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  891.880883] env[74598]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  891.880883] env[74598]: ERROR neutron     response = self.get(path=path, params=params)
[  891.881541] env[74598]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  891.881541] env[74598]: ERROR neutron     raise RuntimeError(msg, last_err)
[  891.881541] env[74598]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  891.881541] env[74598]: ERROR neutron 
[  894.510663] env[74853]: INFO neutron.common.config [-] Logging enabled!
[  894.511047] env[74853]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  894.511195] env[74853]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74853) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  894.511427] env[74853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  894.511516] env[74853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  894.512394] env[74853]: WARNING oslo_policy.policy [None req-f210ce55-0038-41b8-8503-e882784da228 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  894.512606] env[74853]: WARNING oslo_policy.policy [None req-f210ce55-0038-41b8-8503-e882784da228 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  894.513827] env[74853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  894.528692] env[74853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Session token - acquiring
[  894.529551] env[74853]: DEBUG networking_nsxv3.common.locking [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74853) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.529768] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.529897] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.530265] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.539652] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.539911] env[74853]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  894.540082] env[74853]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74853) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  896.546322] env[74853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Session token - acquiring
[  896.546768] env[74853]: DEBUG networking_nsxv3.common.locking [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74853) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.546880] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.547013] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.547311] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.556365] env[74853]: DEBUG oslo_concurrency.lockutils [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74853) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.556600] env[74853]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  896.556747] env[74853]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74853) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  898.559710] env[74853]: CRITICAL neutron [None req-f210ce55-0038-41b8-8503-e882784da228 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  898.559710] env[74853]: ERROR neutron Traceback (most recent call last):
[  898.559710] env[74853]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  898.559710] env[74853]: ERROR neutron     sys.exit(main())
[  898.559710] env[74853]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  898.559710] env[74853]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  898.559710] env[74853]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  898.559710] env[74853]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  898.559710] env[74853]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  898.559710] env[74853]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  898.559710] env[74853]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  898.559710] env[74853]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  898.559710] env[74853]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  898.559710] env[74853]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  898.559710] env[74853]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  898.559710] env[74853]: ERROR neutron     response = self.get(path=path, params=params)
[  898.560357] env[74853]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  898.560357] env[74853]: ERROR neutron     raise RuntimeError(msg, last_err)
[  898.560357] env[74853]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  898.560357] env[74853]: ERROR neutron 
[  901.332532] env[74929]: INFO neutron.common.config [-] Logging enabled!
[  901.332532] env[74929]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  901.333523] env[74929]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74929) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  901.333523] env[74929]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  901.333523] env[74929]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  901.334003] env[74929]: WARNING oslo_policy.policy [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  901.334212] env[74929]: WARNING oslo_policy.policy [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  901.335451] env[74929]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  901.350061] env[74929]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Session token - acquiring
[  901.350425] env[74929]: DEBUG networking_nsxv3.common.locking [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74929) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.350758] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.350758] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.351113] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.360292] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.360886] env[74929]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  901.360886] env[74929]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74929) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  903.369673] env[74929]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Session token - acquiring
[  903.369673] env[74929]: DEBUG networking_nsxv3.common.locking [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=74929) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  903.369673] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  903.370259] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  903.370259] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  903.378937] env[74929]: DEBUG oslo_concurrency.lockutils [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74929) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  903.378937] env[74929]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  903.379096] env[74929]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74929) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  905.380924] env[74929]: CRITICAL neutron [None req-ca1d4aea-284f-40bc-a74f-592170c26945 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  905.380924] env[74929]: ERROR neutron Traceback (most recent call last):
[  905.380924] env[74929]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  905.380924] env[74929]: ERROR neutron     sys.exit(main())
[  905.380924] env[74929]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  905.380924] env[74929]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  905.380924] env[74929]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  905.380924] env[74929]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  905.380924] env[74929]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  905.380924] env[74929]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  905.380924] env[74929]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  905.380924] env[74929]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  905.380924] env[74929]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  905.380924] env[74929]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  905.380924] env[74929]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  905.380924] env[74929]: ERROR neutron     response = self.get(path=path, params=params)
[  905.381684] env[74929]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  905.381684] env[74929]: ERROR neutron     raise RuntimeError(msg, last_err)
[  905.381684] env[74929]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  905.381684] env[74929]: ERROR neutron 
[  908.091344] env[75182]: INFO neutron.common.config [-] Logging enabled!
[  908.091619] env[75182]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  908.091895] env[75182]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75182) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  908.092166] env[75182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  908.092273] env[75182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  908.093244] env[75182]: WARNING oslo_policy.policy [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  908.093481] env[75182]: WARNING oslo_policy.policy [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  908.094778] env[75182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  908.109313] env[75182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Session token - acquiring
[  908.112031] env[75182]: DEBUG networking_nsxv3.common.locking [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75182) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  908.112031] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  908.112235] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  908.112235] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  908.119995] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  908.120344] env[75182]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  908.120556] env[75182]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75182) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  910.126670] env[75182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Session token - acquiring
[  910.127043] env[75182]: DEBUG networking_nsxv3.common.locking [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75182) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  910.127659] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  910.127868] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  910.128295] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  910.137501] env[75182]: DEBUG oslo_concurrency.lockutils [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75182) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  910.138083] env[75182]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  910.138226] env[75182]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75182) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  912.140913] env[75182]: CRITICAL neutron [None req-089574d4-aa0a-4007-ba4c-abd326a7fa71 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  912.140913] env[75182]: ERROR neutron Traceback (most recent call last):
[  912.140913] env[75182]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  912.140913] env[75182]: ERROR neutron     sys.exit(main())
[  912.140913] env[75182]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  912.140913] env[75182]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  912.140913] env[75182]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  912.140913] env[75182]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  912.140913] env[75182]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  912.140913] env[75182]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  912.140913] env[75182]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  912.140913] env[75182]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  912.140913] env[75182]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  912.140913] env[75182]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  912.140913] env[75182]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  912.140913] env[75182]: ERROR neutron     response = self.get(path=path, params=params)
[  912.141484] env[75182]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  912.141484] env[75182]: ERROR neutron     raise RuntimeError(msg, last_err)
[  912.141484] env[75182]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  912.141484] env[75182]: ERROR neutron 
[  914.814578] env[75380]: INFO neutron.common.config [-] Logging enabled!
[  914.814915] env[75380]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  914.815099] env[75380]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75380) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  914.815322] env[75380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  914.815427] env[75380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  914.816273] env[75380]: WARNING oslo_policy.policy [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  914.816487] env[75380]: WARNING oslo_policy.policy [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  914.817700] env[75380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  914.831925] env[75380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Session token - acquiring
[  914.832705] env[75380]: DEBUG networking_nsxv3.common.locking [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75380) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  914.832889] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  914.833010] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  914.833363] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  914.842424] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  914.842661] env[75380]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  914.842811] env[75380]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75380) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  916.849263] env[75380]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Session token - acquiring
[  916.849504] env[75380]: DEBUG networking_nsxv3.common.locking [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75380) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.849651] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.849755] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.850036] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.859165] env[75380]: DEBUG oslo_concurrency.lockutils [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75380) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.859396] env[75380]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  916.859530] env[75380]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75380) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  918.862456] env[75380]: CRITICAL neutron [None req-3e53577f-cb7d-447a-8bea-a2d60cc32286 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  918.862456] env[75380]: ERROR neutron Traceback (most recent call last):
[  918.862456] env[75380]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  918.862456] env[75380]: ERROR neutron     sys.exit(main())
[  918.862456] env[75380]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  918.862456] env[75380]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  918.862456] env[75380]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  918.862456] env[75380]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  918.862456] env[75380]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  918.862456] env[75380]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  918.862456] env[75380]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  918.862456] env[75380]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  918.862456] env[75380]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  918.862456] env[75380]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  918.862456] env[75380]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  918.862456] env[75380]: ERROR neutron     response = self.get(path=path, params=params)
[  918.863329] env[75380]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  918.863329] env[75380]: ERROR neutron     raise RuntimeError(msg, last_err)
[  918.863329] env[75380]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  918.863329] env[75380]: ERROR neutron 
[  921.532757] env[75539]: INFO neutron.common.config [-] Logging enabled!
[  921.533107] env[75539]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  921.533307] env[75539]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75539) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  921.533543] env[75539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  921.533637] env[75539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  921.534531] env[75539]: WARNING oslo_policy.policy [None req-699c21aa-88a8-4726-9921-117d77b347af None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  921.534773] env[75539]: WARNING oslo_policy.policy [None req-699c21aa-88a8-4726-9921-117d77b347af None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  921.536063] env[75539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  921.557430] env[75539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Session token - acquiring
[  921.558334] env[75539]: DEBUG networking_nsxv3.common.locking [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75539) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  921.558543] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  921.558669] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  921.559042] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  921.568743] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.569163] env[75539]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  921.569443] env[75539]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75539) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  923.575687] env[75539]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Session token - acquiring
[  923.576086] env[75539]: DEBUG networking_nsxv3.common.locking [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75539) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.576224] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.576347] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.576745] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.586007] env[75539]: DEBUG oslo_concurrency.lockutils [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75539) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.586240] env[75539]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  923.586381] env[75539]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75539) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  925.588912] env[75539]: CRITICAL neutron [None req-699c21aa-88a8-4726-9921-117d77b347af None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  925.588912] env[75539]: ERROR neutron Traceback (most recent call last):
[  925.588912] env[75539]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  925.588912] env[75539]: ERROR neutron     sys.exit(main())
[  925.588912] env[75539]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  925.588912] env[75539]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  925.588912] env[75539]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  925.588912] env[75539]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  925.588912] env[75539]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  925.588912] env[75539]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  925.588912] env[75539]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  925.588912] env[75539]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  925.588912] env[75539]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  925.588912] env[75539]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  925.588912] env[75539]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  925.588912] env[75539]: ERROR neutron     response = self.get(path=path, params=params)
[  925.589593] env[75539]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  925.589593] env[75539]: ERROR neutron     raise RuntimeError(msg, last_err)
[  925.589593] env[75539]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  925.589593] env[75539]: ERROR neutron 
[  928.261354] env[75581]: INFO neutron.common.config [-] Logging enabled!
[  928.261708] env[75581]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  928.261842] env[75581]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75581) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  928.262061] env[75581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  928.262150] env[75581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  928.262990] env[75581]: WARNING oslo_policy.policy [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  928.263205] env[75581]: WARNING oslo_policy.policy [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  928.264421] env[75581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  928.278441] env[75581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Session token - acquiring
[  928.279229] env[75581]: DEBUG networking_nsxv3.common.locking [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75581) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.279428] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.279557] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.279908] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.289061] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.289306] env[75581]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  928.289467] env[75581]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75581) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  930.296114] env[75581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Session token - acquiring
[  930.297013] env[75581]: DEBUG networking_nsxv3.common.locking [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75581) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  930.297013] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  930.297163] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  930.297163] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  930.306124] env[75581]: DEBUG oslo_concurrency.lockutils [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75581) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  930.306432] env[75581]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  930.306509] env[75581]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75581) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  932.308879] env[75581]: CRITICAL neutron [None req-61ae4db1-45ff-4596-abab-3d4dd9687b05 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  932.308879] env[75581]: ERROR neutron Traceback (most recent call last):
[  932.308879] env[75581]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  932.308879] env[75581]: ERROR neutron     sys.exit(main())
[  932.308879] env[75581]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  932.308879] env[75581]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  932.308879] env[75581]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  932.308879] env[75581]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  932.308879] env[75581]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  932.308879] env[75581]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  932.308879] env[75581]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  932.308879] env[75581]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  932.308879] env[75581]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  932.308879] env[75581]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  932.308879] env[75581]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  932.308879] env[75581]: ERROR neutron     response = self.get(path=path, params=params)
[  932.309460] env[75581]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  932.309460] env[75581]: ERROR neutron     raise RuntimeError(msg, last_err)
[  932.309460] env[75581]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  932.309460] env[75581]: ERROR neutron 
[  935.054496] env[75749]: INFO neutron.common.config [-] Logging enabled!
[  935.054723] env[75749]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  935.054968] env[75749]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75749) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  935.055195] env[75749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  935.055542] env[75749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  935.056487] env[75749]: WARNING oslo_policy.policy [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  935.056710] env[75749]: WARNING oslo_policy.policy [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  935.057934] env[75749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  935.071847] env[75749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Session token - acquiring
[  935.072651] env[75749]: DEBUG networking_nsxv3.common.locking [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75749) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.072856] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.072983] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.073348] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.083463] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.083463] env[75749]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  935.083463] env[75749]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75749) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  937.089208] env[75749]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Session token - acquiring
[  937.089500] env[75749]: DEBUG networking_nsxv3.common.locking [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75749) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  937.089677] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  937.089800] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  937.090073] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  937.099048] env[75749]: DEBUG oslo_concurrency.lockutils [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75749) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  937.099277] env[75749]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  937.099416] env[75749]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75749) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  939.101101] env[75749]: CRITICAL neutron [None req-a3bff3a8-1821-4c3b-a4e9-d04ce50784e5 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  939.101101] env[75749]: ERROR neutron Traceback (most recent call last):
[  939.101101] env[75749]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  939.101101] env[75749]: ERROR neutron     sys.exit(main())
[  939.101101] env[75749]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  939.101101] env[75749]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  939.101101] env[75749]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  939.101101] env[75749]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  939.101101] env[75749]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  939.101101] env[75749]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  939.101101] env[75749]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  939.101101] env[75749]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  939.101101] env[75749]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  939.101101] env[75749]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  939.101101] env[75749]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  939.101101] env[75749]: ERROR neutron     response = self.get(path=path, params=params)
[  939.101743] env[75749]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  939.101743] env[75749]: ERROR neutron     raise RuntimeError(msg, last_err)
[  939.101743] env[75749]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  939.101743] env[75749]: ERROR neutron 
[  941.812301] env[75922]: INFO neutron.common.config [-] Logging enabled!
[  941.812601] env[75922]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  941.813554] env[75922]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75922) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  941.813554] env[75922]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  941.813554] env[75922]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  941.814144] env[75922]: WARNING oslo_policy.policy [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  941.814399] env[75922]: WARNING oslo_policy.policy [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  941.816193] env[75922]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  941.830286] env[75922]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Session token - acquiring
[  941.831185] env[75922]: DEBUG networking_nsxv3.common.locking [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75922) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.831396] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.831525] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.831907] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.841240] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.841357] env[75922]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  941.841512] env[75922]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75922) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  943.847697] env[75922]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Session token - acquiring
[  943.848042] env[75922]: DEBUG networking_nsxv3.common.locking [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=75922) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.848258] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.848392] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.848663] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.858806] env[75922]: DEBUG oslo_concurrency.lockutils [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75922) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.859062] env[75922]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  943.859249] env[75922]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75922) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  945.864081] env[75922]: CRITICAL neutron [None req-cf6b2253-a916-4775-9c9f-5891bf414ab8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  945.864081] env[75922]: ERROR neutron Traceback (most recent call last):
[  945.864081] env[75922]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  945.864081] env[75922]: ERROR neutron     sys.exit(main())
[  945.864081] env[75922]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  945.864081] env[75922]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  945.864081] env[75922]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  945.864081] env[75922]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  945.864081] env[75922]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  945.864081] env[75922]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  945.864081] env[75922]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  945.864081] env[75922]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  945.864081] env[75922]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  945.864081] env[75922]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  945.864081] env[75922]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  945.864081] env[75922]: ERROR neutron     response = self.get(path=path, params=params)
[  945.864956] env[75922]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  945.864956] env[75922]: ERROR neutron     raise RuntimeError(msg, last_err)
[  945.864956] env[75922]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  945.864956] env[75922]: ERROR neutron 
[  948.536718] env[76256]: INFO neutron.common.config [-] Logging enabled!
[  948.537038] env[76256]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  948.537444] env[76256]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76256) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  948.537530] env[76256]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  948.537590] env[76256]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  948.538408] env[76256]: WARNING oslo_policy.policy [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  948.538632] env[76256]: WARNING oslo_policy.policy [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  948.539908] env[76256]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  948.554231] env[76256]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Session token - acquiring
[  948.555084] env[76256]: DEBUG networking_nsxv3.common.locking [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76256) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  948.555288] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  948.555486] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  948.555971] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  948.565776] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  948.566010] env[76256]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  948.566154] env[76256]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76256) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  950.572299] env[76256]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Session token - acquiring
[  950.572673] env[76256]: DEBUG networking_nsxv3.common.locking [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76256) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  950.572783] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  950.572908] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  950.573199] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  950.582913] env[76256]: DEBUG oslo_concurrency.lockutils [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76256) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  950.583465] env[76256]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  950.583780] env[76256]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76256) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  952.585780] env[76256]: CRITICAL neutron [None req-df7f1ef8-fe64-435f-8b67-9829200e7b85 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  952.585780] env[76256]: ERROR neutron Traceback (most recent call last):
[  952.585780] env[76256]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  952.585780] env[76256]: ERROR neutron     sys.exit(main())
[  952.585780] env[76256]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  952.585780] env[76256]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  952.585780] env[76256]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  952.585780] env[76256]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  952.585780] env[76256]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  952.585780] env[76256]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  952.585780] env[76256]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  952.585780] env[76256]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  952.585780] env[76256]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  952.585780] env[76256]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  952.585780] env[76256]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  952.585780] env[76256]: ERROR neutron     response = self.get(path=path, params=params)
[  952.586549] env[76256]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  952.586549] env[76256]: ERROR neutron     raise RuntimeError(msg, last_err)
[  952.586549] env[76256]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  952.586549] env[76256]: ERROR neutron 
[  955.317701] env[76367]: INFO neutron.common.config [-] Logging enabled!
[  955.318027] env[76367]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  955.318254] env[76367]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76367) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  955.318477] env[76367]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  955.318572] env[76367]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  955.319434] env[76367]: WARNING oslo_policy.policy [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  955.319661] env[76367]: WARNING oslo_policy.policy [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  955.320898] env[76367]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  955.335382] env[76367]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Session token - acquiring
[  955.335692] env[76367]: DEBUG networking_nsxv3.common.locking [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76367) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  955.335895] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  955.336155] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  955.336407] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  955.345421] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  955.345658] env[76367]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  955.345804] env[76367]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76367) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  957.353687] env[76367]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Session token - acquiring
[  957.354036] env[76367]: DEBUG networking_nsxv3.common.locking [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76367) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  957.354169] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  957.354295] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  957.354587] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  957.363432] env[76367]: DEBUG oslo_concurrency.lockutils [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76367) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  957.363649] env[76367]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  957.363781] env[76367]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76367) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  959.366525] env[76367]: CRITICAL neutron [None req-aaf867aa-ab1d-4b04-976d-c1a77da03d13 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  959.366525] env[76367]: ERROR neutron Traceback (most recent call last):
[  959.366525] env[76367]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  959.366525] env[76367]: ERROR neutron     sys.exit(main())
[  959.366525] env[76367]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  959.366525] env[76367]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  959.366525] env[76367]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  959.366525] env[76367]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  959.366525] env[76367]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  959.366525] env[76367]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  959.366525] env[76367]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  959.366525] env[76367]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  959.366525] env[76367]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  959.366525] env[76367]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  959.366525] env[76367]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  959.366525] env[76367]: ERROR neutron     response = self.get(path=path, params=params)
[  959.367237] env[76367]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  959.367237] env[76367]: ERROR neutron     raise RuntimeError(msg, last_err)
[  959.367237] env[76367]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  959.367237] env[76367]: ERROR neutron 
[  961.986130] env[76441]: INFO neutron.common.config [-] Logging enabled!
[  961.986431] env[76441]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  961.986654] env[76441]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76441) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  961.986886] env[76441]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  961.986980] env[76441]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  961.987810] env[76441]: WARNING oslo_policy.policy [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  961.988039] env[76441]: WARNING oslo_policy.policy [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  961.989233] env[76441]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  962.003224] env[76441]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Session token - acquiring
[  962.003997] env[76441]: DEBUG networking_nsxv3.common.locking [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76441) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  962.004219] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  962.004357] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  962.004780] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  962.013524] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  962.013768] env[76441]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  962.013917] env[76441]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76441) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  964.020048] env[76441]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Session token - acquiring
[  964.020369] env[76441]: DEBUG networking_nsxv3.common.locking [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76441) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  964.020540] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  964.020662] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  964.020942] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  964.030917] env[76441]: DEBUG oslo_concurrency.lockutils [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76441) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  964.031174] env[76441]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  964.031314] env[76441]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76441) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  966.032952] env[76441]: CRITICAL neutron [None req-f21d4bcb-a73a-4087-b81f-a0eadaaf6086 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  966.032952] env[76441]: ERROR neutron Traceback (most recent call last):
[  966.032952] env[76441]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  966.032952] env[76441]: ERROR neutron     sys.exit(main())
[  966.032952] env[76441]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  966.032952] env[76441]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  966.032952] env[76441]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  966.032952] env[76441]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  966.032952] env[76441]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  966.032952] env[76441]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  966.032952] env[76441]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  966.032952] env[76441]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  966.032952] env[76441]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  966.032952] env[76441]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  966.032952] env[76441]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  966.032952] env[76441]: ERROR neutron     response = self.get(path=path, params=params)
[  966.033974] env[76441]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  966.033974] env[76441]: ERROR neutron     raise RuntimeError(msg, last_err)
[  966.033974] env[76441]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  966.033974] env[76441]: ERROR neutron 
[  968.822500] env[76512]: INFO neutron.common.config [-] Logging enabled!
[  968.822801] env[76512]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  968.823038] env[76512]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76512) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  968.823266] env[76512]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  968.823362] env[76512]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  968.824295] env[76512]: WARNING oslo_policy.policy [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  968.824471] env[76512]: WARNING oslo_policy.policy [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  968.825711] env[76512]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  968.840279] env[76512]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Session token - acquiring
[  968.841094] env[76512]: DEBUG networking_nsxv3.common.locking [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76512) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  968.841301] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  968.841428] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  968.841800] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  968.851517] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  968.851778] env[76512]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  968.851927] env[76512]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76512) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  970.857991] env[76512]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Session token - acquiring
[  970.858384] env[76512]: DEBUG networking_nsxv3.common.locking [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76512) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  970.858573] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  970.858715] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  970.858981] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  970.868246] env[76512]: DEBUG oslo_concurrency.lockutils [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76512) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  970.868484] env[76512]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  970.868626] env[76512]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76512) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  972.871507] env[76512]: CRITICAL neutron [None req-87760e6e-377d-47ed-a6a8-da985114bc27 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  972.871507] env[76512]: ERROR neutron Traceback (most recent call last):
[  972.871507] env[76512]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  972.871507] env[76512]: ERROR neutron     sys.exit(main())
[  972.871507] env[76512]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  972.871507] env[76512]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  972.871507] env[76512]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  972.871507] env[76512]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  972.871507] env[76512]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  972.871507] env[76512]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  972.871507] env[76512]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  972.871507] env[76512]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  972.871507] env[76512]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  972.871507] env[76512]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  972.871507] env[76512]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  972.871507] env[76512]: ERROR neutron     response = self.get(path=path, params=params)
[  972.872248] env[76512]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  972.872248] env[76512]: ERROR neutron     raise RuntimeError(msg, last_err)
[  972.872248] env[76512]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  972.872248] env[76512]: ERROR neutron 
[  975.538962] env[76615]: INFO neutron.common.config [-] Logging enabled!
[  975.539265] env[76615]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  975.539515] env[76615]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76615) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  975.539753] env[76615]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  975.539850] env[76615]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  975.540740] env[76615]: WARNING oslo_policy.policy [None req-292103bb-8676-4773-9fa7-90dea961839e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  975.541016] env[76615]: WARNING oslo_policy.policy [None req-292103bb-8676-4773-9fa7-90dea961839e None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  975.542282] env[76615]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  975.561177] env[76615]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Session token - acquiring
[  975.562070] env[76615]: DEBUG networking_nsxv3.common.locking [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76615) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.562292] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.562463] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.562860] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.574939] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.575214] env[76615]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  975.575391] env[76615]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76615) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  977.581727] env[76615]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Session token - acquiring
[  977.582055] env[76615]: DEBUG networking_nsxv3.common.locking [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76615) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.582055] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.582147] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.582499] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.591602] env[76615]: DEBUG oslo_concurrency.lockutils [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76615) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.591849] env[76615]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  977.591991] env[76615]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76615) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  979.595114] env[76615]: CRITICAL neutron [None req-292103bb-8676-4773-9fa7-90dea961839e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  979.595114] env[76615]: ERROR neutron Traceback (most recent call last):
[  979.595114] env[76615]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  979.595114] env[76615]: ERROR neutron     sys.exit(main())
[  979.595114] env[76615]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  979.595114] env[76615]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  979.595114] env[76615]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  979.595114] env[76615]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  979.595114] env[76615]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  979.595114] env[76615]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  979.595114] env[76615]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  979.595114] env[76615]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  979.595114] env[76615]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  979.595114] env[76615]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  979.595114] env[76615]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  979.595114] env[76615]: ERROR neutron     response = self.get(path=path, params=params)
[  979.595827] env[76615]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  979.595827] env[76615]: ERROR neutron     raise RuntimeError(msg, last_err)
[  979.595827] env[76615]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  979.595827] env[76615]: ERROR neutron 
[  982.246107] env[76758]: INFO neutron.common.config [-] Logging enabled!
[  982.246461] env[76758]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  982.246634] env[76758]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76758) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  982.246896] env[76758]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  982.246993] env[76758]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  982.247824] env[76758]: WARNING oslo_policy.policy [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  982.248054] env[76758]: WARNING oslo_policy.policy [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  982.249366] env[76758]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  982.263206] env[76758]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Session token - acquiring
[  982.264035] env[76758]: DEBUG networking_nsxv3.common.locking [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76758) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  982.264247] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  982.264390] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  982.264773] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  982.273812] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  982.274055] env[76758]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  982.274206] env[76758]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76758) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  984.279976] env[76758]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Session token - acquiring
[  984.280314] env[76758]: DEBUG networking_nsxv3.common.locking [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76758) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  984.280499] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  984.280621] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  984.280897] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  984.290785] env[76758]: DEBUG oslo_concurrency.lockutils [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76758) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  984.290785] env[76758]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  984.290785] env[76758]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76758) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  986.292894] env[76758]: CRITICAL neutron [None req-dbf02ea9-338e-4019-a990-7aa3da92b102 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  986.292894] env[76758]: ERROR neutron Traceback (most recent call last):
[  986.292894] env[76758]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  986.292894] env[76758]: ERROR neutron     sys.exit(main())
[  986.292894] env[76758]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  986.292894] env[76758]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  986.292894] env[76758]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  986.292894] env[76758]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  986.292894] env[76758]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  986.292894] env[76758]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  986.292894] env[76758]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  986.292894] env[76758]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  986.292894] env[76758]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  986.292894] env[76758]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  986.292894] env[76758]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  986.292894] env[76758]: ERROR neutron     response = self.get(path=path, params=params)
[  986.293637] env[76758]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  986.293637] env[76758]: ERROR neutron     raise RuntimeError(msg, last_err)
[  986.293637] env[76758]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  986.293637] env[76758]: ERROR neutron 
[  988.912196] env[76897]: INFO neutron.common.config [-] Logging enabled!
[  988.912484] env[76897]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  988.912683] env[76897]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76897) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  988.912909] env[76897]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  988.913002] env[76897]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  988.913826] env[76897]: WARNING oslo_policy.policy [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  988.914042] env[76897]: WARNING oslo_policy.policy [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  988.915311] env[76897]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  988.929280] env[76897]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Session token - acquiring
[  988.930040] env[76897]: DEBUG networking_nsxv3.common.locking [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76897) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  988.930241] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  988.930366] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  988.930723] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  988.939451] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  988.939671] env[76897]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  988.939838] env[76897]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76897) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  990.946448] env[76897]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Session token - acquiring
[  990.946737] env[76897]: DEBUG networking_nsxv3.common.locking [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=76897) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  990.947339] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  990.947339] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  990.947339] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  990.957428] env[76897]: DEBUG oslo_concurrency.lockutils [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76897) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  990.957830] env[76897]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  990.957890] env[76897]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=76897) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  992.960874] env[76897]: CRITICAL neutron [None req-6ac0c1a5-ac61-445f-8216-7264d9cee53d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  992.960874] env[76897]: ERROR neutron Traceback (most recent call last):
[  992.960874] env[76897]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  992.960874] env[76897]: ERROR neutron     sys.exit(main())
[  992.960874] env[76897]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  992.960874] env[76897]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  992.960874] env[76897]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  992.960874] env[76897]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  992.960874] env[76897]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  992.960874] env[76897]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  992.960874] env[76897]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  992.960874] env[76897]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  992.960874] env[76897]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  992.960874] env[76897]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  992.960874] env[76897]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  992.960874] env[76897]: ERROR neutron     response = self.get(path=path, params=params)
[  992.961516] env[76897]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  992.961516] env[76897]: ERROR neutron     raise RuntimeError(msg, last_err)
[  992.961516] env[76897]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  992.961516] env[76897]: ERROR neutron 
[  995.603687] env[77155]: INFO neutron.common.config [-] Logging enabled!
[  995.604052] env[77155]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[  995.604375] env[77155]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77155) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  995.604728] env[77155]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  995.604833] env[77155]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  995.605821] env[77155]: WARNING oslo_policy.policy [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  995.606097] env[77155]: WARNING oslo_policy.policy [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  995.607509] env[77155]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  995.622550] env[77155]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Session token - acquiring
[  995.623694] env[77155]: DEBUG networking_nsxv3.common.locking [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77155) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.623973] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.624182] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.624736] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.650236] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.650659] env[77155]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  995.650854] env[77155]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77155) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  997.657069] env[77155]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Session token - acquiring
[  997.657408] env[77155]: DEBUG networking_nsxv3.common.locking [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77155) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  997.657547] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  997.657673] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  997.657950] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  997.666845] env[77155]: DEBUG oslo_concurrency.lockutils [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77155) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  997.667073] env[77155]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[  997.667211] env[77155]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77155) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[  999.670169] env[77155]: CRITICAL neutron [None req-85d34a58-7c38-4523-9cd6-d8c66559c0f7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  999.670169] env[77155]: ERROR neutron Traceback (most recent call last):
[  999.670169] env[77155]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[  999.670169] env[77155]: ERROR neutron     sys.exit(main())
[  999.670169] env[77155]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[  999.670169] env[77155]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[  999.670169] env[77155]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[  999.670169] env[77155]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[  999.670169] env[77155]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[  999.670169] env[77155]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[  999.670169] env[77155]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[  999.670169] env[77155]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[  999.670169] env[77155]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[  999.670169] env[77155]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[  999.670169] env[77155]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[  999.670169] env[77155]: ERROR neutron     response = self.get(path=path, params=params)
[  999.670911] env[77155]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[  999.670911] env[77155]: ERROR neutron     raise RuntimeError(msg, last_err)
[  999.670911] env[77155]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[  999.670911] env[77155]: ERROR neutron 
[ 1002.244500] env[77186]: INFO neutron.common.config [-] Logging enabled!
[ 1002.244800] env[77186]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1002.245059] env[77186]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77186) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1002.245223] env[77186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1002.245312] env[77186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1002.246171] env[77186]: WARNING oslo_policy.policy [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1002.246385] env[77186]: WARNING oslo_policy.policy [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1002.247607] env[77186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1002.265172] env[77186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Session token - acquiring
[ 1002.266147] env[77186]: DEBUG networking_nsxv3.common.locking [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77186) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.266353] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.266491] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.266882] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.275932] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.276193] env[77186]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1002.276345] env[77186]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77186) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1004.282418] env[77186]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Session token - acquiring
[ 1004.282969] env[77186]: DEBUG networking_nsxv3.common.locking [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77186) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1004.282969] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1004.283078] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1004.283287] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1004.292306] env[77186]: DEBUG oslo_concurrency.lockutils [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77186) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1004.292551] env[77186]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1004.292692] env[77186]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77186) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1006.295684] env[77186]: CRITICAL neutron [None req-1fc3d11a-1f04-4536-8371-af818022f2fd None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1006.295684] env[77186]: ERROR neutron Traceback (most recent call last):
[ 1006.295684] env[77186]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1006.295684] env[77186]: ERROR neutron     sys.exit(main())
[ 1006.295684] env[77186]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1006.295684] env[77186]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1006.295684] env[77186]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1006.295684] env[77186]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1006.295684] env[77186]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1006.295684] env[77186]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1006.295684] env[77186]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1006.295684] env[77186]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1006.295684] env[77186]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1006.295684] env[77186]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1006.295684] env[77186]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1006.295684] env[77186]: ERROR neutron     response = self.get(path=path, params=params)
[ 1006.296692] env[77186]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1006.296692] env[77186]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1006.296692] env[77186]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1006.296692] env[77186]: ERROR neutron 
[ 1009.003467] env[77215]: INFO neutron.common.config [-] Logging enabled!
[ 1009.003815] env[77215]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1009.003984] env[77215]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77215) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1009.004272] env[77215]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1009.004336] env[77215]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1009.005155] env[77215]: WARNING oslo_policy.policy [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1009.005366] env[77215]: WARNING oslo_policy.policy [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1009.006555] env[77215]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1009.021460] env[77215]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Session token - acquiring
[ 1009.022252] env[77215]: DEBUG networking_nsxv3.common.locking [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77215) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1009.022454] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1009.022581] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1009.022935] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1009.032219] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1009.032472] env[77215]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1009.032620] env[77215]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77215) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1011.038832] env[77215]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Session token - acquiring
[ 1011.039116] env[77215]: DEBUG networking_nsxv3.common.locking [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77215) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1011.039301] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1011.039420] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1011.039728] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1011.049731] env[77215]: DEBUG oslo_concurrency.lockutils [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77215) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1011.049987] env[77215]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1011.050128] env[77215]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77215) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1013.052923] env[77215]: CRITICAL neutron [None req-2f869964-98f1-47ee-a074-f7e63e6c2275 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1013.052923] env[77215]: ERROR neutron Traceback (most recent call last):
[ 1013.052923] env[77215]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1013.052923] env[77215]: ERROR neutron     sys.exit(main())
[ 1013.052923] env[77215]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1013.052923] env[77215]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1013.052923] env[77215]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1013.052923] env[77215]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1013.052923] env[77215]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1013.052923] env[77215]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1013.052923] env[77215]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1013.052923] env[77215]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1013.052923] env[77215]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1013.052923] env[77215]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1013.052923] env[77215]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1013.052923] env[77215]: ERROR neutron     response = self.get(path=path, params=params)
[ 1013.053861] env[77215]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1013.053861] env[77215]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1013.053861] env[77215]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1013.053861] env[77215]: ERROR neutron 
[ 1015.874074] env[77243]: INFO neutron.common.config [-] Logging enabled!
[ 1015.874074] env[77243]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1015.874074] env[77243]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77243) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1015.874461] env[77243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1015.874461] env[77243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1015.875173] env[77243]: WARNING oslo_policy.policy [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1015.875374] env[77243]: WARNING oslo_policy.policy [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1015.876917] env[77243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1015.890753] env[77243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Session token - acquiring
[ 1015.891498] env[77243]: DEBUG networking_nsxv3.common.locking [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77243) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1015.891796] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1015.891873] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1015.892246] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1015.901405] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1015.901662] env[77243]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1015.901810] env[77243]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77243) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1017.912256] env[77243]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Session token - acquiring
[ 1017.912256] env[77243]: DEBUG networking_nsxv3.common.locking [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77243) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1017.912256] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1017.912887] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1017.912887] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1017.920437] env[77243]: DEBUG oslo_concurrency.lockutils [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77243) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1017.920866] env[77243]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1017.921129] env[77243]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77243) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1019.924140] env[77243]: CRITICAL neutron [None req-d0bbf30f-bd7d-451c-a32d-1895d01560a3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1019.924140] env[77243]: ERROR neutron Traceback (most recent call last):
[ 1019.924140] env[77243]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1019.924140] env[77243]: ERROR neutron     sys.exit(main())
[ 1019.924140] env[77243]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1019.924140] env[77243]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1019.924140] env[77243]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1019.924140] env[77243]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1019.924140] env[77243]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1019.924140] env[77243]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1019.924140] env[77243]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1019.924140] env[77243]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1019.924140] env[77243]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1019.924140] env[77243]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1019.924140] env[77243]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1019.924140] env[77243]: ERROR neutron     response = self.get(path=path, params=params)
[ 1019.925076] env[77243]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1019.925076] env[77243]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1019.925076] env[77243]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1019.925076] env[77243]: ERROR neutron 
[ 1022.538824] env[77487]: INFO neutron.common.config [-] Logging enabled!
[ 1022.539272] env[77487]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1022.539606] env[77487]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77487) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1022.540042] env[77487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1022.540162] env[77487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1022.541563] env[77487]: WARNING oslo_policy.policy [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1022.541901] env[77487]: WARNING oslo_policy.policy [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1022.543799] env[77487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1022.558315] env[77487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Session token - acquiring
[ 1022.559154] env[77487]: DEBUG networking_nsxv3.common.locking [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77487) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1022.559380] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1022.559537] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1022.559938] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1022.570049] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1022.570313] env[77487]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1022.570470] env[77487]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77487) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1024.576589] env[77487]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Session token - acquiring
[ 1024.576968] env[77487]: DEBUG networking_nsxv3.common.locking [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77487) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1024.577167] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1024.577292] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1024.577563] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1024.586949] env[77487]: DEBUG oslo_concurrency.lockutils [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77487) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1024.586949] env[77487]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1024.587089] env[77487]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77487) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1026.589017] env[77487]: CRITICAL neutron [None req-b48781b0-869b-4327-9efc-540f2bf68886 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1026.589017] env[77487]: ERROR neutron Traceback (most recent call last):
[ 1026.589017] env[77487]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1026.589017] env[77487]: ERROR neutron     sys.exit(main())
[ 1026.589017] env[77487]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1026.589017] env[77487]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1026.589017] env[77487]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1026.589017] env[77487]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1026.589017] env[77487]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1026.589017] env[77487]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1026.589017] env[77487]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1026.589017] env[77487]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1026.589017] env[77487]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1026.589017] env[77487]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1026.589017] env[77487]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1026.589017] env[77487]: ERROR neutron     response = self.get(path=path, params=params)
[ 1026.589829] env[77487]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1026.589829] env[77487]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1026.589829] env[77487]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1026.589829] env[77487]: ERROR neutron 
[ 1029.321693] env[77629]: INFO neutron.common.config [-] Logging enabled!
[ 1029.322076] env[77629]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1029.322208] env[77629]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77629) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1029.322467] env[77629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1029.322543] env[77629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1029.323375] env[77629]: WARNING oslo_policy.policy [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1029.323589] env[77629]: WARNING oslo_policy.policy [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1029.324818] env[77629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1029.339551] env[77629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Session token - acquiring
[ 1029.340406] env[77629]: DEBUG networking_nsxv3.common.locking [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77629) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1029.340610] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1029.340738] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1029.341112] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1029.350067] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1029.350304] env[77629]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1029.350453] env[77629]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77629) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1031.356525] env[77629]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Session token - acquiring
[ 1031.356826] env[77629]: DEBUG networking_nsxv3.common.locking [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77629) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1031.356985] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1031.357111] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1031.357392] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1031.366233] env[77629]: DEBUG oslo_concurrency.lockutils [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77629) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1031.366462] env[77629]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1031.366606] env[77629]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77629) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1033.368898] env[77629]: CRITICAL neutron [None req-e906387b-fac4-4dc2-9174-9b0ef3d55eb1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1033.368898] env[77629]: ERROR neutron Traceback (most recent call last):
[ 1033.368898] env[77629]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1033.368898] env[77629]: ERROR neutron     sys.exit(main())
[ 1033.368898] env[77629]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1033.368898] env[77629]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1033.368898] env[77629]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1033.368898] env[77629]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1033.368898] env[77629]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1033.368898] env[77629]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1033.368898] env[77629]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1033.368898] env[77629]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1033.368898] env[77629]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1033.368898] env[77629]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1033.368898] env[77629]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1033.368898] env[77629]: ERROR neutron     response = self.get(path=path, params=params)
[ 1033.369636] env[77629]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1033.369636] env[77629]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1033.369636] env[77629]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1033.369636] env[77629]: ERROR neutron 
[ 1035.984019] env[77717]: INFO neutron.common.config [-] Logging enabled!
[ 1035.984370] env[77717]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1035.984549] env[77717]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77717) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1035.984792] env[77717]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1035.984962] env[77717]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1035.985709] env[77717]: WARNING oslo_policy.policy [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1035.986052] env[77717]: WARNING oslo_policy.policy [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1035.987125] env[77717]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1036.007251] env[77717]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Session token - acquiring
[ 1036.008090] env[77717]: DEBUG networking_nsxv3.common.locking [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77717) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1036.008300] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1036.008426] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1036.008804] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1036.018008] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1036.018247] env[77717]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1036.018394] env[77717]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77717) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1038.027452] env[77717]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Session token - acquiring
[ 1038.027452] env[77717]: DEBUG networking_nsxv3.common.locking [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77717) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1038.027452] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1038.028097] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1038.028097] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1038.037573] env[77717]: DEBUG oslo_concurrency.lockutils [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77717) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1038.037573] env[77717]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1038.037573] env[77717]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77717) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1040.039387] env[77717]: CRITICAL neutron [None req-52896413-3fe9-4316-8595-e1cfc7392e24 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1040.039387] env[77717]: ERROR neutron Traceback (most recent call last):
[ 1040.039387] env[77717]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1040.039387] env[77717]: ERROR neutron     sys.exit(main())
[ 1040.039387] env[77717]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1040.039387] env[77717]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1040.039387] env[77717]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1040.039387] env[77717]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1040.039387] env[77717]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1040.039387] env[77717]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1040.039387] env[77717]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1040.039387] env[77717]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1040.039387] env[77717]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1040.039387] env[77717]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1040.039387] env[77717]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1040.039387] env[77717]: ERROR neutron     response = self.get(path=path, params=params)
[ 1040.040401] env[77717]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1040.040401] env[77717]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1040.040401] env[77717]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1040.040401] env[77717]: ERROR neutron 
[ 1042.468016] env[77852]: INFO neutron.common.config [-] Logging enabled!
[ 1042.468355] env[77852]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1042.468549] env[77852]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77852) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1042.468775] env[77852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1042.468870] env[77852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1042.469683] env[77852]: WARNING oslo_policy.policy [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1042.469913] env[77852]: WARNING oslo_policy.policy [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1042.471104] env[77852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1042.487063] env[77852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Session token - acquiring
[ 1042.487829] env[77852]: DEBUG networking_nsxv3.common.locking [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77852) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1042.488049] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1042.488176] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1042.488582] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1042.498023] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1042.498275] env[77852]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1042.498428] env[77852]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77852) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1044.506495] env[77852]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Session token - acquiring
[ 1044.506881] env[77852]: DEBUG networking_nsxv3.common.locking [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77852) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.507058] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.507185] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.507475] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.516838] env[77852]: DEBUG oslo_concurrency.lockutils [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77852) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.517091] env[77852]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1044.517254] env[77852]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77852) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1046.520176] env[77852]: CRITICAL neutron [None req-683ae149-e015-4fe9-af93-add977e2a411 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1046.520176] env[77852]: ERROR neutron Traceback (most recent call last):
[ 1046.520176] env[77852]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1046.520176] env[77852]: ERROR neutron     sys.exit(main())
[ 1046.520176] env[77852]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1046.520176] env[77852]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1046.520176] env[77852]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1046.520176] env[77852]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1046.520176] env[77852]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1046.520176] env[77852]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1046.520176] env[77852]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1046.520176] env[77852]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1046.520176] env[77852]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1046.520176] env[77852]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1046.520176] env[77852]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1046.520176] env[77852]: ERROR neutron     response = self.get(path=path, params=params)
[ 1046.520835] env[77852]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1046.520835] env[77852]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1046.520835] env[77852]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1046.520835] env[77852]: ERROR neutron 
[ 1049.237823] env[77913]: INFO neutron.common.config [-] Logging enabled!
[ 1049.238182] env[77913]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1049.238333] env[77913]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77913) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1049.238552] env[77913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1049.238635] env[77913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1049.239450] env[77913]: WARNING oslo_policy.policy [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1049.239667] env[77913]: WARNING oslo_policy.policy [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1049.240878] env[77913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1049.253856] env[77913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Session token - acquiring
[ 1049.254663] env[77913]: DEBUG networking_nsxv3.common.locking [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77913) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1049.254853] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1049.254978] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1049.255330] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1049.264497] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1049.264742] env[77913]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1049.264892] env[77913]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77913) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1051.270913] env[77913]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Session token - acquiring
[ 1051.271302] env[77913]: DEBUG networking_nsxv3.common.locking [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77913) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1051.271386] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1051.271519] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1051.271840] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1051.280797] env[77913]: DEBUG oslo_concurrency.lockutils [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77913) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1051.281040] env[77913]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1051.281179] env[77913]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77913) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1053.282126] env[77913]: CRITICAL neutron [None req-63b3fe36-2ce7-444a-88c0-d291f858a986 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1053.282126] env[77913]: ERROR neutron Traceback (most recent call last):
[ 1053.282126] env[77913]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1053.282126] env[77913]: ERROR neutron     sys.exit(main())
[ 1053.282126] env[77913]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1053.282126] env[77913]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1053.282126] env[77913]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1053.282126] env[77913]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1053.282126] env[77913]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1053.282126] env[77913]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1053.282126] env[77913]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1053.282126] env[77913]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1053.282126] env[77913]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1053.282126] env[77913]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1053.282126] env[77913]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1053.282126] env[77913]: ERROR neutron     response = self.get(path=path, params=params)
[ 1053.282859] env[77913]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1053.282859] env[77913]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1053.282859] env[77913]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1053.282859] env[77913]: ERROR neutron 
[ 1055.719632] env[77975]: INFO neutron.common.config [-] Logging enabled!
[ 1055.719929] env[77975]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1055.720185] env[77975]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=77975) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1055.720440] env[77975]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1055.720542] env[77975]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1055.721385] env[77975]: WARNING oslo_policy.policy [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1055.721613] env[77975]: WARNING oslo_policy.policy [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1055.722856] env[77975]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1055.747303] env[77975]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Session token - acquiring
[ 1055.748208] env[77975]: DEBUG networking_nsxv3.common.locking [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77975) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1055.748432] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1055.748566] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1055.748951] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1055.766416] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1055.766729] env[77975]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1055.766886] env[77975]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=77975) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1057.772993] env[77975]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Session token - acquiring
[ 1057.773393] env[77975]: DEBUG networking_nsxv3.common.locking [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=77975) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1057.773518] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1057.773648] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1057.773941] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1057.783174] env[77975]: DEBUG oslo_concurrency.lockutils [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=77975) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1057.783413] env[77975]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1057.783560] env[77975]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=77975) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1059.786545] env[77975]: CRITICAL neutron [None req-c00190df-b794-40d0-9f0e-c4592663d504 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1059.786545] env[77975]: ERROR neutron Traceback (most recent call last):
[ 1059.786545] env[77975]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1059.786545] env[77975]: ERROR neutron     sys.exit(main())
[ 1059.786545] env[77975]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1059.786545] env[77975]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1059.786545] env[77975]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1059.786545] env[77975]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1059.786545] env[77975]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1059.786545] env[77975]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1059.786545] env[77975]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1059.786545] env[77975]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1059.786545] env[77975]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1059.786545] env[77975]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1059.786545] env[77975]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1059.786545] env[77975]: ERROR neutron     response = self.get(path=path, params=params)
[ 1059.787405] env[77975]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1059.787405] env[77975]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1059.787405] env[77975]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1059.787405] env[77975]: ERROR neutron 
[ 1062.177665] env[78002]: INFO neutron.common.config [-] Logging enabled!
[ 1062.178049] env[78002]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1062.178196] env[78002]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78002) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1062.178423] env[78002]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1062.178511] env[78002]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1062.179361] env[78002]: WARNING oslo_policy.policy [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1062.179574] env[78002]: WARNING oslo_policy.policy [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1062.180852] env[78002]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1062.195337] env[78002]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Session token - acquiring
[ 1062.195834] env[78002]: DEBUG networking_nsxv3.common.locking [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78002) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1062.196056] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1062.196188] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1062.196553] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1062.205691] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1062.205930] env[78002]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1062.206079] env[78002]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78002) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1064.212391] env[78002]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Session token - acquiring
[ 1064.212926] env[78002]: DEBUG networking_nsxv3.common.locking [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78002) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1064.213314] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1064.213419] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1064.213757] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1064.223483] env[78002]: DEBUG oslo_concurrency.lockutils [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78002) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1064.223625] env[78002]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1064.223688] env[78002]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78002) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1066.225008] env[78002]: CRITICAL neutron [None req-fc414a28-507b-41d8-abe2-2b971c163916 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1066.225008] env[78002]: ERROR neutron Traceback (most recent call last):
[ 1066.225008] env[78002]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1066.225008] env[78002]: ERROR neutron     sys.exit(main())
[ 1066.225008] env[78002]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1066.225008] env[78002]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1066.225008] env[78002]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1066.225008] env[78002]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1066.225008] env[78002]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1066.225008] env[78002]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1066.225008] env[78002]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1066.225008] env[78002]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1066.225008] env[78002]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1066.225008] env[78002]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1066.225008] env[78002]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1066.225008] env[78002]: ERROR neutron     response = self.get(path=path, params=params)
[ 1066.225784] env[78002]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1066.225784] env[78002]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1066.225784] env[78002]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1066.225784] env[78002]: ERROR neutron 
[ 1068.683181] env[78072]: INFO neutron.common.config [-] Logging enabled!
[ 1068.683536] env[78072]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1068.683709] env[78072]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78072) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1068.683945] env[78072]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1068.684096] env[78072]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1068.684982] env[78072]: WARNING oslo_policy.policy [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1068.685202] env[78072]: WARNING oslo_policy.policy [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1068.686470] env[78072]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1068.700777] env[78072]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Session token - acquiring
[ 1068.701551] env[78072]: DEBUG networking_nsxv3.common.locking [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78072) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.701756] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.701900] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.702253] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.711175] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.711412] env[78072]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1068.711560] env[78072]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78072) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1070.717693] env[78072]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Session token - acquiring
[ 1070.718058] env[78072]: DEBUG networking_nsxv3.common.locking [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78072) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1070.718193] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1070.718328] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1070.718608] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1070.727691] env[78072]: DEBUG oslo_concurrency.lockutils [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78072) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1070.727917] env[78072]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1070.728070] env[78072]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78072) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1072.730264] env[78072]: CRITICAL neutron [None req-5a784092-8841-49e6-a378-ffa9fb0a2ecc None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1072.730264] env[78072]: ERROR neutron Traceback (most recent call last):
[ 1072.730264] env[78072]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1072.730264] env[78072]: ERROR neutron     sys.exit(main())
[ 1072.730264] env[78072]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1072.730264] env[78072]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1072.730264] env[78072]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1072.730264] env[78072]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1072.730264] env[78072]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1072.730264] env[78072]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1072.730264] env[78072]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1072.730264] env[78072]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1072.730264] env[78072]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1072.730264] env[78072]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1072.730264] env[78072]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1072.730264] env[78072]: ERROR neutron     response = self.get(path=path, params=params)
[ 1072.731274] env[78072]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1072.731274] env[78072]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1072.731274] env[78072]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1072.731274] env[78072]: ERROR neutron 
[ 1075.221289] env[78102]: INFO neutron.common.config [-] Logging enabled!
[ 1075.221739] env[78102]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1075.221806] env[78102]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78102) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1075.222012] env[78102]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1075.222108] env[78102]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1075.222955] env[78102]: WARNING oslo_policy.policy [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1075.223173] env[78102]: WARNING oslo_policy.policy [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1075.224351] env[78102]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1075.237993] env[78102]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Session token - acquiring
[ 1075.238748] env[78102]: DEBUG networking_nsxv3.common.locking [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78102) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1075.238950] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1075.239077] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1075.239432] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1075.248518] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1075.249068] env[78102]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1075.249068] env[78102]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78102) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1077.255013] env[78102]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Session token - acquiring
[ 1077.255337] env[78102]: DEBUG networking_nsxv3.common.locking [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78102) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1077.255519] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1077.255650] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1077.255996] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1077.265607] env[78102]: DEBUG oslo_concurrency.lockutils [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78102) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1077.265832] env[78102]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1077.265972] env[78102]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78102) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1079.268861] env[78102]: CRITICAL neutron [None req-92113f0a-29f8-4a5c-ae98-9106a4215c4c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1079.268861] env[78102]: ERROR neutron Traceback (most recent call last):
[ 1079.268861] env[78102]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1079.268861] env[78102]: ERROR neutron     sys.exit(main())
[ 1079.268861] env[78102]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1079.268861] env[78102]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1079.268861] env[78102]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1079.268861] env[78102]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1079.268861] env[78102]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1079.268861] env[78102]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1079.268861] env[78102]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1079.268861] env[78102]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1079.268861] env[78102]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1079.268861] env[78102]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1079.268861] env[78102]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1079.268861] env[78102]: ERROR neutron     response = self.get(path=path, params=params)
[ 1079.269553] env[78102]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1079.269553] env[78102]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1079.269553] env[78102]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1079.269553] env[78102]: ERROR neutron 
[ 1082.022538] env[78152]: INFO neutron.common.config [-] Logging enabled!
[ 1082.022538] env[78152]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1082.022924] env[78152]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78152) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1082.023181] env[78152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1082.023269] env[78152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1082.024144] env[78152]: WARNING oslo_policy.policy [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1082.024360] env[78152]: WARNING oslo_policy.policy [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1082.025598] env[78152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1082.039723] env[78152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Session token - acquiring
[ 1082.040506] env[78152]: DEBUG networking_nsxv3.common.locking [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78152) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1082.040710] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1082.040840] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1082.041200] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1082.049941] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1082.050179] env[78152]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1082.050329] env[78152]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78152) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1084.056349] env[78152]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Session token - acquiring
[ 1084.056640] env[78152]: DEBUG networking_nsxv3.common.locking [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78152) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1084.056895] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1084.057044] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1084.057416] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1084.066310] env[78152]: DEBUG oslo_concurrency.lockutils [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78152) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1084.066533] env[78152]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1084.066672] env[78152]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78152) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1086.068912] env[78152]: CRITICAL neutron [None req-ecba2e01-b694-454d-b28c-9863b7195d40 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1086.068912] env[78152]: ERROR neutron Traceback (most recent call last):
[ 1086.068912] env[78152]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1086.068912] env[78152]: ERROR neutron     sys.exit(main())
[ 1086.068912] env[78152]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1086.068912] env[78152]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1086.068912] env[78152]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1086.068912] env[78152]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1086.068912] env[78152]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1086.068912] env[78152]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1086.068912] env[78152]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1086.068912] env[78152]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1086.068912] env[78152]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1086.068912] env[78152]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1086.068912] env[78152]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1086.068912] env[78152]: ERROR neutron     response = self.get(path=path, params=params)
[ 1086.070073] env[78152]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1086.070073] env[78152]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1086.070073] env[78152]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1086.070073] env[78152]: ERROR neutron 
[ 1088.678893] env[78183]: INFO neutron.common.config [-] Logging enabled!
[ 1088.679419] env[78183]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1088.679419] env[78183]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78183) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1088.679756] env[78183]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1088.679756] env[78183]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1088.680657] env[78183]: WARNING oslo_policy.policy [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1088.680875] env[78183]: WARNING oslo_policy.policy [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1088.682514] env[78183]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1088.700379] env[78183]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Session token - acquiring
[ 1088.701569] env[78183]: DEBUG networking_nsxv3.common.locking [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78183) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1088.701797] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1088.701922] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1088.702294] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1088.711351] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1088.711595] env[78183]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1088.711747] env[78183]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78183) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1090.717889] env[78183]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Session token - acquiring
[ 1090.718235] env[78183]: DEBUG networking_nsxv3.common.locking [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78183) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1090.718413] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1090.718547] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1090.718825] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1090.727857] env[78183]: DEBUG oslo_concurrency.lockutils [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78183) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1090.728091] env[78183]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1090.728231] env[78183]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78183) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1092.731353] env[78183]: CRITICAL neutron [None req-11c8f833-d2e6-4c86-9ecb-3a73ead9da03 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1092.731353] env[78183]: ERROR neutron Traceback (most recent call last):
[ 1092.731353] env[78183]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1092.731353] env[78183]: ERROR neutron     sys.exit(main())
[ 1092.731353] env[78183]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1092.731353] env[78183]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1092.731353] env[78183]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1092.731353] env[78183]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1092.731353] env[78183]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1092.731353] env[78183]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1092.731353] env[78183]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1092.731353] env[78183]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1092.731353] env[78183]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1092.731353] env[78183]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1092.731353] env[78183]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1092.731353] env[78183]: ERROR neutron     response = self.get(path=path, params=params)
[ 1092.732210] env[78183]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1092.732210] env[78183]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1092.732210] env[78183]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1092.732210] env[78183]: ERROR neutron 
[ 1095.267075] env[78252]: INFO neutron.common.config [-] Logging enabled!
[ 1095.267346] env[78252]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1095.267622] env[78252]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78252) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1095.267856] env[78252]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1095.267974] env[78252]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1095.268826] env[78252]: WARNING oslo_policy.policy [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1095.269040] env[78252]: WARNING oslo_policy.policy [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1095.270896] env[78252]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1095.285703] env[78252]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Session token - acquiring
[ 1095.286512] env[78252]: DEBUG networking_nsxv3.common.locking [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78252) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1095.286747] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1095.286851] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1095.287236] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1095.299882] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1095.300195] env[78252]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1095.300390] env[78252]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78252) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1097.307375] env[78252]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Session token - acquiring
[ 1097.307671] env[78252]: DEBUG networking_nsxv3.common.locking [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78252) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1097.307864] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1097.307989] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1097.308302] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1097.322879] env[78252]: DEBUG oslo_concurrency.lockutils [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78252) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1097.323133] env[78252]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1097.323274] env[78252]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78252) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1099.325878] env[78252]: CRITICAL neutron [None req-9a3128d2-0ad3-4104-aea7-d22593e2eae4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1099.325878] env[78252]: ERROR neutron Traceback (most recent call last):
[ 1099.325878] env[78252]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1099.325878] env[78252]: ERROR neutron     sys.exit(main())
[ 1099.325878] env[78252]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1099.325878] env[78252]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1099.325878] env[78252]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1099.325878] env[78252]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1099.325878] env[78252]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1099.325878] env[78252]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1099.325878] env[78252]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1099.325878] env[78252]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1099.325878] env[78252]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1099.325878] env[78252]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1099.325878] env[78252]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1099.325878] env[78252]: ERROR neutron     response = self.get(path=path, params=params)
[ 1099.326689] env[78252]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1099.326689] env[78252]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1099.326689] env[78252]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1099.326689] env[78252]: ERROR neutron 
[ 1101.977958] env[78634]: INFO neutron.common.config [-] Logging enabled!
[ 1101.978254] env[78634]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1101.978501] env[78634]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78634) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1101.978727] env[78634]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1101.978817] env[78634]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1101.979654] env[78634]: WARNING oslo_policy.policy [None req-3d6b453e-224b-4591-865f-8120115db992 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1101.979867] env[78634]: WARNING oslo_policy.policy [None req-3d6b453e-224b-4591-865f-8120115db992 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1101.981103] env[78634]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1101.995336] env[78634]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Session token - acquiring
[ 1101.996148] env[78634]: DEBUG networking_nsxv3.common.locking [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78634) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1101.996437] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1101.996569] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1101.996933] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1102.006582] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1102.006826] env[78634]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1102.006975] env[78634]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78634) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1104.014078] env[78634]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Session token - acquiring
[ 1104.014423] env[78634]: DEBUG networking_nsxv3.common.locking [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78634) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1104.014671] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1104.014796] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1104.015076] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1104.024281] env[78634]: DEBUG oslo_concurrency.lockutils [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78634) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1104.024589] env[78634]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1104.024730] env[78634]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78634) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1106.027934] env[78634]: CRITICAL neutron [None req-3d6b453e-224b-4591-865f-8120115db992 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1106.027934] env[78634]: ERROR neutron Traceback (most recent call last):
[ 1106.027934] env[78634]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1106.027934] env[78634]: ERROR neutron     sys.exit(main())
[ 1106.027934] env[78634]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1106.027934] env[78634]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1106.027934] env[78634]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1106.027934] env[78634]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1106.027934] env[78634]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1106.027934] env[78634]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1106.027934] env[78634]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1106.027934] env[78634]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1106.027934] env[78634]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1106.027934] env[78634]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1106.027934] env[78634]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1106.027934] env[78634]: ERROR neutron     response = self.get(path=path, params=params)
[ 1106.029068] env[78634]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1106.029068] env[78634]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1106.029068] env[78634]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1106.029068] env[78634]: ERROR neutron 
[ 1108.376895] env[78743]: INFO neutron.common.config [-] Logging enabled!
[ 1108.377505] env[78743]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1108.377995] env[78743]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78743) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1108.378337] env[78743]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1108.378537] env[78743]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1108.379475] env[78743]: WARNING oslo_policy.policy [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1108.379824] env[78743]: WARNING oslo_policy.policy [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1108.381114] env[78743]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1108.395457] env[78743]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Session token - acquiring
[ 1108.396336] env[78743]: DEBUG networking_nsxv3.common.locking [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78743) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1108.396685] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1108.396939] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1108.397385] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1108.406086] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1108.406453] env[78743]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1108.406718] env[78743]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78743) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1110.412964] env[78743]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Session token - acquiring
[ 1110.413338] env[78743]: DEBUG networking_nsxv3.common.locking [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78743) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1110.413543] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1110.413669] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1110.413937] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1110.422667] env[78743]: DEBUG oslo_concurrency.lockutils [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78743) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1110.422894] env[78743]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1110.423030] env[78743]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78743) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1112.424925] env[78743]: CRITICAL neutron [None req-7205b0cb-4659-4832-8cf6-af69fa084069 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1112.424925] env[78743]: ERROR neutron Traceback (most recent call last):
[ 1112.424925] env[78743]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1112.424925] env[78743]: ERROR neutron     sys.exit(main())
[ 1112.424925] env[78743]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1112.424925] env[78743]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1112.424925] env[78743]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1112.424925] env[78743]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1112.424925] env[78743]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1112.424925] env[78743]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1112.424925] env[78743]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1112.424925] env[78743]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1112.424925] env[78743]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1112.424925] env[78743]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1112.424925] env[78743]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1112.424925] env[78743]: ERROR neutron     response = self.get(path=path, params=params)
[ 1112.425683] env[78743]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1112.425683] env[78743]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1112.425683] env[78743]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1112.425683] env[78743]: ERROR neutron 
[ 1114.884618] env[78792]: INFO neutron.common.config [-] Logging enabled!
[ 1114.885012] env[78792]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1114.885142] env[78792]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78792) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1114.885378] env[78792]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1114.885918] env[78792]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1114.886300] env[78792]: WARNING oslo_policy.policy [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1114.886519] env[78792]: WARNING oslo_policy.policy [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1114.887744] env[78792]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1114.902061] env[78792]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Session token - acquiring
[ 1114.902885] env[78792]: DEBUG networking_nsxv3.common.locking [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78792) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1114.903084] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1114.903230] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1114.903652] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1114.913029] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1114.913268] env[78792]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1114.913418] env[78792]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78792) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1116.919517] env[78792]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Session token - acquiring
[ 1116.919908] env[78792]: DEBUG networking_nsxv3.common.locking [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78792) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1116.920126] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1116.920258] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1116.920529] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1116.929727] env[78792]: DEBUG oslo_concurrency.lockutils [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78792) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1116.929954] env[78792]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1116.930092] env[78792]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78792) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1118.933080] env[78792]: CRITICAL neutron [None req-c135ced1-1958-4ff6-9f05-cd7a59092935 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1118.933080] env[78792]: ERROR neutron Traceback (most recent call last):
[ 1118.933080] env[78792]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1118.933080] env[78792]: ERROR neutron     sys.exit(main())
[ 1118.933080] env[78792]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1118.933080] env[78792]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1118.933080] env[78792]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1118.933080] env[78792]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1118.933080] env[78792]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1118.933080] env[78792]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1118.933080] env[78792]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1118.933080] env[78792]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1118.933080] env[78792]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1118.933080] env[78792]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1118.933080] env[78792]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1118.933080] env[78792]: ERROR neutron     response = self.get(path=path, params=params)
[ 1118.934449] env[78792]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1118.934449] env[78792]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1118.934449] env[78792]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1118.934449] env[78792]: ERROR neutron 
[ 1121.366648] env[78816]: INFO neutron.common.config [-] Logging enabled!
[ 1121.367107] env[78816]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1121.367180] env[78816]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78816) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1121.367357] env[78816]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1121.367450] env[78816]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1121.368291] env[78816]: WARNING oslo_policy.policy [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1121.368511] env[78816]: WARNING oslo_policy.policy [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1121.369658] env[78816]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1121.383605] env[78816]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Session token - acquiring
[ 1121.384348] env[78816]: DEBUG networking_nsxv3.common.locking [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78816) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1121.384565] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1121.384699] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1121.385053] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1121.393586] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1121.393824] env[78816]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1121.393972] env[78816]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78816) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1123.400344] env[78816]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Session token - acquiring
[ 1123.400773] env[78816]: DEBUG networking_nsxv3.common.locking [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78816) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1123.400988] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1123.401117] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1123.401385] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1123.410139] env[78816]: DEBUG oslo_concurrency.lockutils [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78816) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1123.410364] env[78816]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1123.410505] env[78816]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78816) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1125.412950] env[78816]: CRITICAL neutron [None req-321cf262-a921-43b6-a2d5-09e3c497d147 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1125.412950] env[78816]: ERROR neutron Traceback (most recent call last):
[ 1125.412950] env[78816]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1125.412950] env[78816]: ERROR neutron     sys.exit(main())
[ 1125.412950] env[78816]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1125.412950] env[78816]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1125.412950] env[78816]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1125.412950] env[78816]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1125.412950] env[78816]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1125.412950] env[78816]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1125.412950] env[78816]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1125.412950] env[78816]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1125.412950] env[78816]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1125.412950] env[78816]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1125.412950] env[78816]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1125.412950] env[78816]: ERROR neutron     response = self.get(path=path, params=params)
[ 1125.414192] env[78816]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1125.414192] env[78816]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1125.414192] env[78816]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1125.414192] env[78816]: ERROR neutron 
[ 1127.855826] env[78826]: INFO neutron.common.config [-] Logging enabled!
[ 1127.856263] env[78826]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1127.856358] env[78826]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78826) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1127.856546] env[78826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1127.856640] env[78826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1127.857452] env[78826]: WARNING oslo_policy.policy [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1127.857668] env[78826]: WARNING oslo_policy.policy [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1127.858806] env[78826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1127.873240] env[78826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Session token - acquiring
[ 1127.873962] env[78826]: DEBUG networking_nsxv3.common.locking [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78826) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1127.874153] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1127.874277] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1127.874653] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1127.883288] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1127.883519] env[78826]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1127.883671] env[78826]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78826) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1129.889732] env[78826]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Session token - acquiring
[ 1129.891334] env[78826]: DEBUG networking_nsxv3.common.locking [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78826) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1129.891334] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1129.891511] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1129.891511] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1129.899555] env[78826]: DEBUG oslo_concurrency.lockutils [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78826) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1129.899802] env[78826]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1129.899942] env[78826]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78826) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1131.900901] env[78826]: CRITICAL neutron [None req-df20ec6c-fa9b-4a09-a5bd-4064ecb9546f None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1131.900901] env[78826]: ERROR neutron Traceback (most recent call last):
[ 1131.900901] env[78826]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1131.900901] env[78826]: ERROR neutron     sys.exit(main())
[ 1131.900901] env[78826]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1131.900901] env[78826]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1131.900901] env[78826]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1131.900901] env[78826]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1131.900901] env[78826]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1131.900901] env[78826]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1131.900901] env[78826]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1131.900901] env[78826]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1131.900901] env[78826]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1131.900901] env[78826]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1131.900901] env[78826]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1131.900901] env[78826]: ERROR neutron     response = self.get(path=path, params=params)
[ 1131.902128] env[78826]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1131.902128] env[78826]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1131.902128] env[78826]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1131.902128] env[78826]: ERROR neutron 
[ 1134.229255] env[78902]: INFO neutron.common.config [-] Logging enabled!
[ 1134.229664] env[78902]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 0.0.1.dev55
[ 1134.229780] env[78902]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=78902) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[ 1134.230000] env[78902]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[ 1134.230097] env[78902]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[ 1134.230899] env[78902]: WARNING oslo_policy.policy [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1134.231118] env[78902]: WARNING oslo_policy.policy [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[ 1134.232291] env[78902]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Looking for TransportZone with name nsx-vlan-transportzone.
[ 1134.246263] env[78902]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Session token - acquiring
[ 1134.247001] env[78902]: DEBUG networking_nsxv3.common.locking [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78902) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1134.247196] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1134.247324] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1134.247671] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1134.256575] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1134.256808] env[78902]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1134.256956] env[78902]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=78902) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1136.262980] env[78902]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Session token - acquiring
[ 1136.263337] env[78902]: DEBUG networking_nsxv3.common.locking [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=78902) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1136.263402] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1136.263504] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1136.263762] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1136.272677] env[78902]: DEBUG oslo_concurrency.lockutils [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=78902) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1136.272903] env[78902]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create
[ 1136.273043] env[78902]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=78902) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}
[ 1138.276023] env[78902]: CRITICAL neutron [None req-b701c611-2499-48c3-a986-3b6c66549321 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1138.276023] env[78902]: ERROR neutron Traceback (most recent call last):
[ 1138.276023] env[78902]: ERROR neutron   File "/opt/stack/data/venv/bin/neutron-nsxv3-agent", line 10, in <module>
[ 1138.276023] env[78902]: ERROR neutron     sys.exit(main())
[ 1138.276023] env[78902]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main
[ 1138.276023] env[78902]: ERROR neutron     NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()),
[ 1138.276023] env[78902]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__
[ 1138.276023] env[78902]: ERROR neutron     self.mngr_provider = provider_nsx_mgmt.Provider()
[ 1138.276023] env[78902]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__
[ 1138.276023] env[78902]: ERROR neutron     super(Provider, self).__init__(client=Client(), zone_id=None)
[ 1138.276023] env[78902]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__
[ 1138.276023] env[78902]: ERROR neutron     self.zone_id, self.zone_tags = self._load_zones()
[ 1138.276023] env[78902]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones
[ 1138.276023] env[78902]: ERROR neutron     for zone in self.client.get_all(path="/api/v1/transport-zones"):
[ 1138.276023] env[78902]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all
[ 1138.276023] env[78902]: ERROR neutron     response = self.get(path=path, params=params)
[ 1138.276916] env[78902]: ERROR neutron   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator
[ 1138.276916] env[78902]: ERROR neutron     raise RuntimeError(msg, last_err)
[ 1138.276916] env[78902]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create'))
[ 1138.276916] env[78902]: ERROR neutron