[ 509.594104] env[59395]: INFO neutron.common.config [-] Logging enabled! [ 509.594690] env[59395]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 509.594690] env[59395]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=59395) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 509.594871] env[59395]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 509.594871] env[59395]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 509.595696] env[59395]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 509.608825] env[59395]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Session token - acquiring [ 509.609549] env[59395]: DEBUG networking_nsxv3.common.locking [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59395) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 509.609755] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 509.609886] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 509.610282] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 509.619658] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 509.619906] env[59395]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 509.620077] env[59395]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=59395) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 511.628302] env[59395]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Session token - acquiring [ 511.628676] env[59395]: DEBUG networking_nsxv3.common.locking [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59395) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 511.628921] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 511.629099] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 511.629444] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 511.639304] env[59395]: DEBUG oslo_concurrency.lockutils [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59395) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 511.639583] env[59395]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 511.639786] env[59395]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=59395) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 513.640879] env[59395]: CRITICAL neutron [None req-1c12ca69-1cca-4823-adb0-43e57f09e739 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 513.640879] env[59395]: ERROR neutron Traceback (most recent call last): [ 513.640879] env[59395]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 513.640879] env[59395]: ERROR neutron sys.exit(main()) [ 513.640879] env[59395]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 513.640879] env[59395]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 513.640879] env[59395]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 513.640879] env[59395]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 513.640879] env[59395]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 513.640879] env[59395]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 513.640879] env[59395]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 513.640879] env[59395]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 513.640879] env[59395]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 513.640879] env[59395]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 513.640879] env[59395]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 513.640879] env[59395]: ERROR neutron response = self.get(path=path, params=params) [ 513.641518] env[59395]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 513.641518] env[59395]: ERROR neutron raise RuntimeError(msg, last_err) [ 513.641518] env[59395]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 513.641518] env[59395]: ERROR neutron [ 516.172555] env[59430]: INFO neutron.common.config [-] Logging enabled! [ 516.172956] env[59430]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 516.173078] env[59430]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=59430) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 516.173276] env[59430]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 516.173370] env[59430]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 516.174220] env[59430]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 516.187815] env[59430]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Session token - acquiring [ 516.188713] env[59430]: DEBUG networking_nsxv3.common.locking [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59430) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 516.188950] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 516.189061] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 516.189424] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 516.199146] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 516.199464] env[59430]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 516.199653] env[59430]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=59430) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 518.206299] env[59430]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Session token - acquiring [ 518.206644] env[59430]: DEBUG networking_nsxv3.common.locking [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59430) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 518.206844] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 518.206970] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 518.207240] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 518.217010] env[59430]: DEBUG oslo_concurrency.lockutils [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59430) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 518.217243] env[59430]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 518.217426] env[59430]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=59430) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 520.220342] env[59430]: CRITICAL neutron [None req-b229441d-c242-4899-aed5-aa5081ddabe3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 520.220342] env[59430]: ERROR neutron Traceback (most recent call last): [ 520.220342] env[59430]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 520.220342] env[59430]: ERROR neutron sys.exit(main()) [ 520.220342] env[59430]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 520.220342] env[59430]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 520.220342] env[59430]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 520.220342] env[59430]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 520.220342] env[59430]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 520.220342] env[59430]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 520.220342] env[59430]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 520.220342] env[59430]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 520.220342] env[59430]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 520.220342] env[59430]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 520.220342] env[59430]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 520.220342] env[59430]: ERROR neutron response = self.get(path=path, params=params) [ 520.221114] env[59430]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 520.221114] env[59430]: ERROR neutron raise RuntimeError(msg, last_err) [ 520.221114] env[59430]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 520.221114] env[59430]: ERROR neutron [ 522.649443] env[59446]: INFO neutron.common.config [-] Logging enabled! [ 522.649831] env[59446]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 522.649979] env[59446]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=59446) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 522.650227] env[59446]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 522.650327] env[59446]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 522.651194] env[59446]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 522.664775] env[59446]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Session token - acquiring [ 522.665545] env[59446]: DEBUG networking_nsxv3.common.locking [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59446) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 522.665752] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 522.665913] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 522.666230] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 522.675494] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 522.675764] env[59446]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 522.675934] env[59446]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=59446) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 524.681862] env[59446]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Session token - acquiring [ 524.682217] env[59446]: DEBUG networking_nsxv3.common.locking [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59446) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 524.682381] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 524.682457] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 524.682740] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 524.692051] env[59446]: DEBUG oslo_concurrency.lockutils [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59446) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 524.692188] env[59446]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 524.692240] env[59446]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=59446) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 526.695153] env[59446]: CRITICAL neutron [None req-b2f04c4a-8c14-4ddb-b610-80d9d1180725 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 526.695153] env[59446]: ERROR neutron Traceback (most recent call last): [ 526.695153] env[59446]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 526.695153] env[59446]: ERROR neutron sys.exit(main()) [ 526.695153] env[59446]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 526.695153] env[59446]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 526.695153] env[59446]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 526.695153] env[59446]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 526.695153] env[59446]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 526.695153] env[59446]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 526.695153] env[59446]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 526.695153] env[59446]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 526.695153] env[59446]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 526.695153] env[59446]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 526.695153] env[59446]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 526.695153] env[59446]: ERROR neutron response = self.get(path=path, params=params) [ 526.695936] env[59446]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 526.695936] env[59446]: ERROR neutron raise RuntimeError(msg, last_err) [ 526.695936] env[59446]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 526.695936] env[59446]: ERROR neutron [ 529.278225] env[59483]: INFO neutron.common.config [-] Logging enabled! [ 529.278621] env[59483]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 529.278660] env[59483]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=59483) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 529.278870] env[59483]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 529.278967] env[59483]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 529.279821] env[59483]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 529.293257] env[59483]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Session token - acquiring [ 529.293969] env[59483]: DEBUG networking_nsxv3.common.locking [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59483) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 529.294168] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 529.294296] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 529.294649] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 529.303452] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 529.303682] env[59483]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 529.303841] env[59483]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=59483) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 531.309807] env[59483]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Session token - acquiring [ 531.310153] env[59483]: DEBUG networking_nsxv3.common.locking [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59483) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 531.310219] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 531.310346] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 531.310628] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 531.320245] env[59483]: DEBUG oslo_concurrency.lockutils [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59483) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 531.320477] env[59483]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 531.320620] env[59483]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=59483) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 533.323134] env[59483]: CRITICAL neutron [None req-0ee03895-eb1a-4956-a2a2-4319e42713f7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 533.323134] env[59483]: ERROR neutron Traceback (most recent call last): [ 533.323134] env[59483]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 533.323134] env[59483]: ERROR neutron sys.exit(main()) [ 533.323134] env[59483]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 533.323134] env[59483]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 533.323134] env[59483]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 533.323134] env[59483]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 533.323134] env[59483]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 533.323134] env[59483]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 533.323134] env[59483]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 533.323134] env[59483]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 533.323134] env[59483]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 533.323134] env[59483]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 533.323134] env[59483]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 533.323134] env[59483]: ERROR neutron response = self.get(path=path, params=params) [ 533.323865] env[59483]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 533.323865] env[59483]: ERROR neutron raise RuntimeError(msg, last_err) [ 533.323865] env[59483]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 533.323865] env[59483]: ERROR neutron [ 535.648014] env[59492]: INFO neutron.common.config [-] Logging enabled! [ 535.648455] env[59492]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 535.648567] env[59492]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=59492) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 535.648779] env[59492]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 535.648875] env[59492]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 535.649699] env[59492]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 535.663057] env[59492]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Session token - acquiring [ 535.663761] env[59492]: DEBUG networking_nsxv3.common.locking [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59492) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 535.663959] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 535.664101] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 535.664447] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 535.673257] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 535.673482] env[59492]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 535.673625] env[59492]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=59492) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 537.681546] env[59492]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Session token - acquiring [ 537.681904] env[59492]: DEBUG networking_nsxv3.common.locking [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59492) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 537.682028] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 537.682155] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 537.682445] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 537.691535] env[59492]: DEBUG oslo_concurrency.lockutils [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59492) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 537.691791] env[59492]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 537.691926] env[59492]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=59492) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 539.692871] env[59492]: CRITICAL neutron [None req-5bbfc1ca-a648-49bd-9fd9-4a4fc3b28c72 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 539.692871] env[59492]: ERROR neutron Traceback (most recent call last): [ 539.692871] env[59492]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 539.692871] env[59492]: ERROR neutron sys.exit(main()) [ 539.692871] env[59492]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 539.692871] env[59492]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 539.692871] env[59492]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 539.692871] env[59492]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 539.692871] env[59492]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 539.692871] env[59492]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 539.692871] env[59492]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 539.692871] env[59492]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 539.692871] env[59492]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 539.692871] env[59492]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 539.692871] env[59492]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 539.692871] env[59492]: ERROR neutron response = self.get(path=path, params=params) [ 539.693495] env[59492]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 539.693495] env[59492]: ERROR neutron raise RuntimeError(msg, last_err) [ 539.693495] env[59492]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 539.693495] env[59492]: ERROR neutron [ 542.160293] env[59579]: INFO neutron.common.config [-] Logging enabled! [ 542.160731] env[59579]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 542.160773] env[59579]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=59579) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 542.161014] env[59579]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 542.161106] env[59579]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 542.161990] env[59579]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 542.175538] env[59579]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Session token - acquiring [ 542.176352] env[59579]: DEBUG networking_nsxv3.common.locking [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59579) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 542.176556] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 542.176685] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 542.177040] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 542.186117] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 542.186385] env[59579]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 542.186544] env[59579]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=59579) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 544.193992] env[59579]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Session token - acquiring [ 544.194329] env[59579]: DEBUG networking_nsxv3.common.locking [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59579) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 544.194461] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 544.194583] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 544.194866] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 544.204881] env[59579]: DEBUG oslo_concurrency.lockutils [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59579) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 544.205167] env[59579]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 544.205309] env[59579]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=59579) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 546.208228] env[59579]: CRITICAL neutron [None req-6edb36fb-eece-4ba6-9d10-1bd8b1784b27 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 546.208228] env[59579]: ERROR neutron Traceback (most recent call last): [ 546.208228] env[59579]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 546.208228] env[59579]: ERROR neutron sys.exit(main()) [ 546.208228] env[59579]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 546.208228] env[59579]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 546.208228] env[59579]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 546.208228] env[59579]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 546.208228] env[59579]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 546.208228] env[59579]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 546.208228] env[59579]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 546.208228] env[59579]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 546.208228] env[59579]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 546.208228] env[59579]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 546.208228] env[59579]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 546.208228] env[59579]: ERROR neutron response = self.get(path=path, params=params) [ 546.208852] env[59579]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 546.208852] env[59579]: ERROR neutron raise RuntimeError(msg, last_err) [ 546.208852] env[59579]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 546.208852] env[59579]: ERROR neutron [ 548.760642] env[59729]: INFO neutron.common.config [-] Logging enabled! [ 548.760941] env[59729]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 548.761139] env[59729]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=59729) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 548.761363] env[59729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 548.761454] env[59729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 548.762332] env[59729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 548.775905] env[59729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Session token - acquiring [ 548.776758] env[59729]: DEBUG networking_nsxv3.common.locking [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59729) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 548.776952] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 548.777076] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 548.777476] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 548.786873] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 548.787186] env[59729]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 548.787383] env[59729]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=59729) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 550.794326] env[59729]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Session token - acquiring [ 550.794326] env[59729]: DEBUG networking_nsxv3.common.locking [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=59729) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 550.794765] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 550.794765] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 550.794765] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 550.807592] env[59729]: DEBUG oslo_concurrency.lockutils [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=59729) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 550.807899] env[59729]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 550.808258] env[59729]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=59729) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 552.811175] env[59729]: CRITICAL neutron [None req-5ccc77b9-63d0-4891-ae19-70662ae9a951 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 552.811175] env[59729]: ERROR neutron Traceback (most recent call last): [ 552.811175] env[59729]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 552.811175] env[59729]: ERROR neutron sys.exit(main()) [ 552.811175] env[59729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 552.811175] env[59729]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 552.811175] env[59729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 552.811175] env[59729]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 552.811175] env[59729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 552.811175] env[59729]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 552.811175] env[59729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 552.811175] env[59729]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 552.811175] env[59729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 552.811175] env[59729]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 552.811175] env[59729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 552.811175] env[59729]: ERROR neutron response = self.get(path=path, params=params) [ 552.811883] env[59729]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 552.811883] env[59729]: ERROR neutron raise RuntimeError(msg, last_err) [ 552.811883] env[59729]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 552.811883] env[59729]: ERROR neutron [ 555.548321] env[60100]: INFO neutron.common.config [-] Logging enabled! [ 555.548597] env[60100]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 555.548940] env[60100]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=60100) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 555.549219] env[60100]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 555.549339] env[60100]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 555.550244] env[60100]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 555.574378] env[60100]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Session token - acquiring [ 555.575277] env[60100]: DEBUG networking_nsxv3.common.locking [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=60100) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 555.575526] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 555.575655] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 555.576055] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 555.586119] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 555.586391] env[60100]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 555.586553] env[60100]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=60100) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 557.593840] env[60100]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Session token - acquiring [ 557.594216] env[60100]: DEBUG networking_nsxv3.common.locking [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=60100) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 557.594357] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 557.594478] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 557.594760] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 557.605379] env[60100]: DEBUG oslo_concurrency.lockutils [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60100) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 557.605379] env[60100]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 557.605379] env[60100]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=60100) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 559.608269] env[60100]: CRITICAL neutron [None req-9ff10633-25a4-43cf-911a-d73e09d2b914 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 559.608269] env[60100]: ERROR neutron Traceback (most recent call last): [ 559.608269] env[60100]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 559.608269] env[60100]: ERROR neutron sys.exit(main()) [ 559.608269] env[60100]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 559.608269] env[60100]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 559.608269] env[60100]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 559.608269] env[60100]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 559.608269] env[60100]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 559.608269] env[60100]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 559.608269] env[60100]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 559.608269] env[60100]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 559.608269] env[60100]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 559.608269] env[60100]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 559.608269] env[60100]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 559.608269] env[60100]: ERROR neutron response = self.get(path=path, params=params) [ 559.608893] env[60100]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 559.608893] env[60100]: ERROR neutron raise RuntimeError(msg, last_err) [ 559.608893] env[60100]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 559.608893] env[60100]: ERROR neutron [ 562.440592] env[60840]: INFO neutron.common.config [-] Logging enabled! [ 562.440834] env[60840]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 562.441108] env[60840]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=60840) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 562.441340] env[60840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 562.441432] env[60840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 562.442457] env[60840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 562.456662] env[60840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Session token - acquiring [ 562.457529] env[60840]: DEBUG networking_nsxv3.common.locking [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=60840) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 562.457747] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 562.457869] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 562.458227] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 562.467691] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 562.468034] env[60840]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 562.468203] env[60840]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=60840) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 564.476172] env[60840]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Session token - acquiring [ 564.476530] env[60840]: DEBUG networking_nsxv3.common.locking [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=60840) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 564.476704] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 564.476803] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 564.477068] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 564.486955] env[60840]: DEBUG oslo_concurrency.lockutils [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60840) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 564.487208] env[60840]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 564.487365] env[60840]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=60840) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 566.488893] env[60840]: CRITICAL neutron [None req-4af0f108-a298-4fa3-aaad-875820aa7976 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 566.488893] env[60840]: ERROR neutron Traceback (most recent call last): [ 566.488893] env[60840]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 566.488893] env[60840]: ERROR neutron sys.exit(main()) [ 566.488893] env[60840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 566.488893] env[60840]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 566.488893] env[60840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 566.488893] env[60840]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 566.488893] env[60840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 566.488893] env[60840]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 566.488893] env[60840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 566.488893] env[60840]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 566.488893] env[60840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 566.488893] env[60840]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 566.488893] env[60840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 566.488893] env[60840]: ERROR neutron response = self.get(path=path, params=params) [ 566.489489] env[60840]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 566.489489] env[60840]: ERROR neutron raise RuntimeError(msg, last_err) [ 566.489489] env[60840]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 566.489489] env[60840]: ERROR neutron [ 569.281020] env[61467]: INFO neutron.common.config [-] Logging enabled! [ 569.281262] env[61467]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 569.281539] env[61467]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=61467) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 569.281778] env[61467]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 569.281876] env[61467]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 569.282761] env[61467]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 569.296852] env[61467]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Session token - acquiring [ 569.297746] env[61467]: DEBUG networking_nsxv3.common.locking [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61467) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 569.297944] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 569.298068] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 569.298439] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 569.309179] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 569.309659] env[61467]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 569.309736] env[61467]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=61467) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 571.317582] env[61467]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Session token - acquiring [ 571.317891] env[61467]: DEBUG networking_nsxv3.common.locking [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=61467) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 571.318091] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 571.318211] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 571.318560] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 571.339243] env[61467]: DEBUG oslo_concurrency.lockutils [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=61467) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 571.339339] env[61467]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 571.339694] env[61467]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=61467) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 573.340849] env[61467]: CRITICAL neutron [None req-c09dec59-fd4b-444a-a700-2a0502fcb768 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 573.340849] env[61467]: ERROR neutron Traceback (most recent call last): [ 573.340849] env[61467]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 573.340849] env[61467]: ERROR neutron sys.exit(main()) [ 573.340849] env[61467]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 573.340849] env[61467]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 573.340849] env[61467]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 573.340849] env[61467]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 573.340849] env[61467]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 573.340849] env[61467]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 573.340849] env[61467]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 573.340849] env[61467]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 573.340849] env[61467]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 573.340849] env[61467]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 573.340849] env[61467]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 573.340849] env[61467]: ERROR neutron response = self.get(path=path, params=params) [ 573.341447] env[61467]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 573.341447] env[61467]: ERROR neutron raise RuntimeError(msg, last_err) [ 573.341447] env[61467]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 573.341447] env[61467]: ERROR neutron [ 576.124207] env[62182]: INFO neutron.common.config [-] Logging enabled! [ 576.124486] env[62182]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 576.124714] env[62182]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62182) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 576.124945] env[62182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 576.125036] env[62182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 576.126106] env[62182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 576.141787] env[62182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Session token - acquiring [ 576.142644] env[62182]: DEBUG networking_nsxv3.common.locking [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62182) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 576.143412] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 576.143590] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 576.143976] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 576.154565] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 576.154912] env[62182]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 576.155077] env[62182]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62182) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 578.162057] env[62182]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Session token - acquiring [ 578.162359] env[62182]: DEBUG networking_nsxv3.common.locking [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62182) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 578.162542] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 578.162662] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 578.162940] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 578.174178] env[62182]: DEBUG oslo_concurrency.lockutils [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62182) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 578.174178] env[62182]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 578.174178] env[62182]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62182) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 580.176778] env[62182]: CRITICAL neutron [None req-ada5ae24-8213-4014-b1eb-3dfa73c0c8d6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 580.176778] env[62182]: ERROR neutron Traceback (most recent call last): [ 580.176778] env[62182]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 580.176778] env[62182]: ERROR neutron sys.exit(main()) [ 580.176778] env[62182]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 580.176778] env[62182]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 580.176778] env[62182]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 580.176778] env[62182]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 580.176778] env[62182]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 580.176778] env[62182]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 580.176778] env[62182]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 580.176778] env[62182]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 580.176778] env[62182]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 580.176778] env[62182]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 580.176778] env[62182]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 580.176778] env[62182]: ERROR neutron response = self.get(path=path, params=params) [ 580.177397] env[62182]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 580.177397] env[62182]: ERROR neutron raise RuntimeError(msg, last_err) [ 580.177397] env[62182]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 580.177397] env[62182]: ERROR neutron [ 582.786739] env[62738]: INFO neutron.common.config [-] Logging enabled! [ 582.787029] env[62738]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 582.787568] env[62738]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=62738) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 582.787741] env[62738]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 582.787833] env[62738]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 582.788862] env[62738]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 582.803579] env[62738]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Session token - acquiring [ 582.804446] env[62738]: DEBUG networking_nsxv3.common.locking [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62738) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 582.804661] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 582.804780] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 582.805144] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 582.825311] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 582.825625] env[62738]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 582.825800] env[62738]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=62738) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 584.833630] env[62738]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Session token - acquiring [ 584.833940] env[62738]: DEBUG networking_nsxv3.common.locking [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=62738) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 584.834134] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 584.834260] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 584.834562] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 584.844163] env[62738]: DEBUG oslo_concurrency.lockutils [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=62738) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 584.844491] env[62738]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 584.844647] env[62738]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=62738) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 586.847699] env[62738]: CRITICAL neutron [None req-a49756f0-758c-4b31-b196-5dd9b72939c0 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 586.847699] env[62738]: ERROR neutron Traceback (most recent call last): [ 586.847699] env[62738]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 586.847699] env[62738]: ERROR neutron sys.exit(main()) [ 586.847699] env[62738]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 586.847699] env[62738]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 586.847699] env[62738]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 586.847699] env[62738]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 586.847699] env[62738]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 586.847699] env[62738]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 586.847699] env[62738]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 586.847699] env[62738]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 586.847699] env[62738]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 586.847699] env[62738]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 586.847699] env[62738]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 586.847699] env[62738]: ERROR neutron response = self.get(path=path, params=params) [ 586.848486] env[62738]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 586.848486] env[62738]: ERROR neutron raise RuntimeError(msg, last_err) [ 586.848486] env[62738]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 586.848486] env[62738]: ERROR neutron [ 589.800991] env[63425]: INFO neutron.common.config [-] Logging enabled! [ 589.801232] env[63425]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 589.801494] env[63425]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=63425) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 589.801723] env[63425]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 589.801813] env[63425]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 589.802693] env[63425]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 589.820747] env[63425]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Session token - acquiring [ 589.821606] env[63425]: DEBUG networking_nsxv3.common.locking [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63425) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 589.821811] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 589.821935] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 589.822296] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 589.840135] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 589.840381] env[63425]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 589.840526] env[63425]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=63425) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 591.848293] env[63425]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Session token - acquiring [ 591.848618] env[63425]: DEBUG networking_nsxv3.common.locking [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=63425) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 591.849096] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 591.849096] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 591.849253] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 591.860388] env[63425]: DEBUG oslo_concurrency.lockutils [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=63425) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 591.860872] env[63425]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 591.860872] env[63425]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=63425) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 593.863518] env[63425]: CRITICAL neutron [None req-67cd0d3d-dd35-4cd0-832c-4f5f6b7efbea None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 593.863518] env[63425]: ERROR neutron Traceback (most recent call last): [ 593.863518] env[63425]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 593.863518] env[63425]: ERROR neutron sys.exit(main()) [ 593.863518] env[63425]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 593.863518] env[63425]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 593.863518] env[63425]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 593.863518] env[63425]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 593.863518] env[63425]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 593.863518] env[63425]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 593.863518] env[63425]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 593.863518] env[63425]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 593.863518] env[63425]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 593.863518] env[63425]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 593.863518] env[63425]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 593.863518] env[63425]: ERROR neutron response = self.get(path=path, params=params) [ 593.864872] env[63425]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 593.864872] env[63425]: ERROR neutron raise RuntimeError(msg, last_err) [ 593.864872] env[63425]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 593.864872] env[63425]: ERROR neutron [ 596.671260] env[64054]: INFO neutron.common.config [-] Logging enabled! [ 596.671594] env[64054]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 596.671781] env[64054]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64054) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 596.672087] env[64054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 596.672198] env[64054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 596.673156] env[64054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 596.687137] env[64054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Session token - acquiring [ 596.688353] env[64054]: DEBUG networking_nsxv3.common.locking [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64054) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 596.688595] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 596.688726] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 596.689105] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 596.699043] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 596.699321] env[64054]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 596.699482] env[64054]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64054) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 598.705714] env[64054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Session token - acquiring [ 598.706064] env[64054]: DEBUG networking_nsxv3.common.locking [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64054) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 598.706255] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 598.706389] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 598.706660] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 598.716552] env[64054]: DEBUG oslo_concurrency.lockutils [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 598.716785] env[64054]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 598.716921] env[64054]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64054) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 600.718019] env[64054]: CRITICAL neutron [None req-18f679ac-b147-4887-b55c-ffc4d172cea1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 600.718019] env[64054]: ERROR neutron Traceback (most recent call last): [ 600.718019] env[64054]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 600.718019] env[64054]: ERROR neutron sys.exit(main()) [ 600.718019] env[64054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 600.718019] env[64054]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 600.718019] env[64054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 600.718019] env[64054]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 600.718019] env[64054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 600.718019] env[64054]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 600.718019] env[64054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 600.718019] env[64054]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 600.718019] env[64054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 600.718019] env[64054]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 600.718019] env[64054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 600.718019] env[64054]: ERROR neutron response = self.get(path=path, params=params) [ 600.718771] env[64054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 600.718771] env[64054]: ERROR neutron raise RuntimeError(msg, last_err) [ 600.718771] env[64054]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 600.718771] env[64054]: ERROR neutron [ 603.568599] env[64663]: INFO neutron.common.config [-] Logging enabled! [ 603.568884] env[64663]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 603.569117] env[64663]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=64663) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 603.569346] env[64663]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 603.569440] env[64663]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 603.570302] env[64663]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 603.583978] env[64663]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Session token - acquiring [ 603.584793] env[64663]: DEBUG networking_nsxv3.common.locking [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64663) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 603.585016] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 603.585160] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 603.585520] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 603.594938] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 603.595215] env[64663]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 603.595440] env[64663]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=64663) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 605.601885] env[64663]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Session token - acquiring [ 605.602189] env[64663]: DEBUG networking_nsxv3.common.locking [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=64663) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 605.602375] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 605.602583] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 605.602783] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 605.614003] env[64663]: DEBUG oslo_concurrency.lockutils [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=64663) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 605.614815] env[64663]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 605.614982] env[64663]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=64663) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 607.619026] env[64663]: CRITICAL neutron [None req-cdb91aca-5510-42a0-85e1-ed6f4ab62a3c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 607.619026] env[64663]: ERROR neutron Traceback (most recent call last): [ 607.619026] env[64663]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 607.619026] env[64663]: ERROR neutron sys.exit(main()) [ 607.619026] env[64663]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 607.619026] env[64663]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 607.619026] env[64663]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 607.619026] env[64663]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 607.619026] env[64663]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 607.619026] env[64663]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 607.619026] env[64663]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 607.619026] env[64663]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 607.619026] env[64663]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 607.619026] env[64663]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 607.619026] env[64663]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 607.619026] env[64663]: ERROR neutron response = self.get(path=path, params=params) [ 607.619798] env[64663]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 607.619798] env[64663]: ERROR neutron raise RuntimeError(msg, last_err) [ 607.619798] env[64663]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 607.619798] env[64663]: ERROR neutron [ 610.509657] env[65258]: INFO neutron.common.config [-] Logging enabled! [ 610.510128] env[65258]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 610.510634] env[65258]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65258) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 610.510998] env[65258]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 610.511201] env[65258]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 610.512255] env[65258]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 610.526949] env[65258]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Session token - acquiring [ 610.528152] env[65258]: DEBUG networking_nsxv3.common.locking [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65258) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 610.528545] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 610.528800] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 610.529290] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 610.544355] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 610.544732] env[65258]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 610.544889] env[65258]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65258) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 612.552659] env[65258]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Session token - acquiring [ 612.552957] env[65258]: DEBUG networking_nsxv3.common.locking [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65258) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 612.553143] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 612.553268] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 612.553544] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 612.562938] env[65258]: DEBUG oslo_concurrency.lockutils [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65258) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 612.563202] env[65258]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 612.563355] env[65258]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65258) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 614.565126] env[65258]: CRITICAL neutron [None req-5a604929-ab2a-4df1-8a22-62f515245e67 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 614.565126] env[65258]: ERROR neutron Traceback (most recent call last): [ 614.565126] env[65258]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 614.565126] env[65258]: ERROR neutron sys.exit(main()) [ 614.565126] env[65258]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 614.565126] env[65258]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 614.565126] env[65258]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 614.565126] env[65258]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 614.565126] env[65258]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 614.565126] env[65258]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 614.565126] env[65258]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 614.565126] env[65258]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 614.565126] env[65258]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 614.565126] env[65258]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 614.565126] env[65258]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 614.565126] env[65258]: ERROR neutron response = self.get(path=path, params=params) [ 614.565806] env[65258]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 614.565806] env[65258]: ERROR neutron raise RuntimeError(msg, last_err) [ 614.565806] env[65258]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 614.565806] env[65258]: ERROR neutron [ 617.353555] env[65904]: INFO neutron.common.config [-] Logging enabled! [ 617.353834] env[65904]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 617.354068] env[65904]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=65904) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 617.354302] env[65904]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 617.354396] env[65904]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 617.355284] env[65904]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 617.369187] env[65904]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Session token - acquiring [ 617.370035] env[65904]: DEBUG networking_nsxv3.common.locking [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65904) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 617.370244] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 617.370372] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 617.370747] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 617.380819] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 617.380819] env[65904]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 617.380819] env[65904]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=65904) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 619.388663] env[65904]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Session token - acquiring [ 619.389002] env[65904]: DEBUG networking_nsxv3.common.locking [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=65904) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 619.389172] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 619.389287] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 619.389608] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 619.404402] env[65904]: DEBUG oslo_concurrency.lockutils [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=65904) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 619.404915] env[65904]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 619.405012] env[65904]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=65904) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 621.407602] env[65904]: CRITICAL neutron [None req-d048176a-8870-4f07-a4c8-d5d6a161c6ec None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 621.407602] env[65904]: ERROR neutron Traceback (most recent call last): [ 621.407602] env[65904]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 621.407602] env[65904]: ERROR neutron sys.exit(main()) [ 621.407602] env[65904]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 621.407602] env[65904]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 621.407602] env[65904]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 621.407602] env[65904]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 621.407602] env[65904]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 621.407602] env[65904]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 621.407602] env[65904]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 621.407602] env[65904]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 621.407602] env[65904]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 621.407602] env[65904]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 621.407602] env[65904]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 621.407602] env[65904]: ERROR neutron response = self.get(path=path, params=params) [ 621.408477] env[65904]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 621.408477] env[65904]: ERROR neutron raise RuntimeError(msg, last_err) [ 621.408477] env[65904]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 621.408477] env[65904]: ERROR neutron [ 624.127201] env[66556]: INFO neutron.common.config [-] Logging enabled! [ 624.127501] env[66556]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 624.127692] env[66556]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=66556) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 624.127934] env[66556]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 624.128040] env[66556]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 624.128981] env[66556]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 624.143089] env[66556]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Session token - acquiring [ 624.143849] env[66556]: DEBUG networking_nsxv3.common.locking [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66556) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 624.144074] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 624.144201] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 624.144587] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 624.156682] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 624.157015] env[66556]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 624.157100] env[66556]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=66556) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 626.164898] env[66556]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Session token - acquiring [ 626.165217] env[66556]: DEBUG networking_nsxv3.common.locking [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=66556) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 626.165410] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 626.165532] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 626.165813] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 626.175943] env[66556]: DEBUG oslo_concurrency.lockutils [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=66556) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 626.176264] env[66556]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 626.176435] env[66556]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=66556) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 628.179305] env[66556]: CRITICAL neutron [None req-d9576483-bb4f-4485-b54c-de0639fba425 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 628.179305] env[66556]: ERROR neutron Traceback (most recent call last): [ 628.179305] env[66556]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 628.179305] env[66556]: ERROR neutron sys.exit(main()) [ 628.179305] env[66556]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 628.179305] env[66556]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 628.179305] env[66556]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 628.179305] env[66556]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 628.179305] env[66556]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 628.179305] env[66556]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 628.179305] env[66556]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 628.179305] env[66556]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 628.179305] env[66556]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 628.179305] env[66556]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 628.179305] env[66556]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 628.179305] env[66556]: ERROR neutron response = self.get(path=path, params=params) [ 628.180109] env[66556]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 628.180109] env[66556]: ERROR neutron raise RuntimeError(msg, last_err) [ 628.180109] env[66556]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 628.180109] env[66556]: ERROR neutron [ 630.736723] env[67270]: INFO neutron.common.config [-] Logging enabled! [ 630.737203] env[67270]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 630.737203] env[67270]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67270) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 630.737413] env[67270]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 630.737553] env[67270]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 630.738314] env[67270]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 630.751597] env[67270]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Session token - acquiring [ 630.752258] env[67270]: DEBUG networking_nsxv3.common.locking [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67270) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 630.752456] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 630.752615] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 630.752969] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 630.762673] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 630.762820] env[67270]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 630.762992] env[67270]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67270) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 632.771519] env[67270]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Session token - acquiring [ 632.771876] env[67270]: DEBUG networking_nsxv3.common.locking [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67270) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 632.772165] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 632.772273] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 632.772542] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 632.782946] env[67270]: DEBUG oslo_concurrency.lockutils [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67270) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 632.783266] env[67270]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 632.783435] env[67270]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67270) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 634.796269] env[67270]: CRITICAL neutron [None req-7880eb7c-378f-4361-b093-094e81111fb9 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 634.796269] env[67270]: ERROR neutron Traceback (most recent call last): [ 634.796269] env[67270]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 634.796269] env[67270]: ERROR neutron sys.exit(main()) [ 634.796269] env[67270]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 634.796269] env[67270]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 634.796269] env[67270]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 634.796269] env[67270]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 634.796269] env[67270]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 634.796269] env[67270]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 634.796269] env[67270]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 634.796269] env[67270]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 634.796269] env[67270]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 634.796269] env[67270]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 634.796269] env[67270]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 634.796269] env[67270]: ERROR neutron response = self.get(path=path, params=params) [ 634.797034] env[67270]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 634.797034] env[67270]: ERROR neutron raise RuntimeError(msg, last_err) [ 634.797034] env[67270]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 634.797034] env[67270]: ERROR neutron [ 637.658742] env[67658]: INFO neutron.common.config [-] Logging enabled! [ 637.659566] env[67658]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 637.659566] env[67658]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=67658) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 637.659566] env[67658]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 637.659665] env[67658]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 637.660529] env[67658]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 637.681436] env[67658]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Session token - acquiring [ 637.682333] env[67658]: DEBUG networking_nsxv3.common.locking [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67658) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 637.682546] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 637.682671] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 637.683043] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 637.700823] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 637.701148] env[67658]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 637.701295] env[67658]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=67658) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 639.709405] env[67658]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Session token - acquiring [ 639.709720] env[67658]: DEBUG networking_nsxv3.common.locking [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=67658) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 639.709908] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 639.710036] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 639.710320] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 639.720276] env[67658]: DEBUG oslo_concurrency.lockutils [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=67658) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 639.720718] env[67658]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 639.720811] env[67658]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=67658) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 641.723732] env[67658]: CRITICAL neutron [None req-9f2fe574-a178-4591-891c-55e95c7cb013 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 641.723732] env[67658]: ERROR neutron Traceback (most recent call last): [ 641.723732] env[67658]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 641.723732] env[67658]: ERROR neutron sys.exit(main()) [ 641.723732] env[67658]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 641.723732] env[67658]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 641.723732] env[67658]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 641.723732] env[67658]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 641.723732] env[67658]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 641.723732] env[67658]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 641.723732] env[67658]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 641.723732] env[67658]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 641.723732] env[67658]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 641.723732] env[67658]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 641.723732] env[67658]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 641.723732] env[67658]: ERROR neutron response = self.get(path=path, params=params) [ 641.724369] env[67658]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 641.724369] env[67658]: ERROR neutron raise RuntimeError(msg, last_err) [ 641.724369] env[67658]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 641.724369] env[67658]: ERROR neutron [ 644.434807] env[68241]: INFO neutron.common.config [-] Logging enabled! [ 644.435070] env[68241]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 644.435344] env[68241]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=68241) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 644.435580] env[68241]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 644.435677] env[68241]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 644.436778] env[68241]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 644.463247] env[68241]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Session token - acquiring [ 644.464211] env[68241]: DEBUG networking_nsxv3.common.locking [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68241) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 644.464440] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 644.464567] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 644.464933] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 644.475844] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 644.476082] env[68241]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 644.476247] env[68241]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=68241) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 646.485896] env[68241]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Session token - acquiring [ 646.485896] env[68241]: DEBUG networking_nsxv3.common.locking [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=68241) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 646.486262] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 646.486262] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 646.486262] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 646.495837] env[68241]: DEBUG oslo_concurrency.lockutils [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=68241) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 646.496440] env[68241]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 646.496838] env[68241]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=68241) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 648.500116] env[68241]: CRITICAL neutron [None req-b82b99db-f175-482e-acc3-d99390e53360 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 648.500116] env[68241]: ERROR neutron Traceback (most recent call last): [ 648.500116] env[68241]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 648.500116] env[68241]: ERROR neutron sys.exit(main()) [ 648.500116] env[68241]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 648.500116] env[68241]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 648.500116] env[68241]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 648.500116] env[68241]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 648.500116] env[68241]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 648.500116] env[68241]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 648.500116] env[68241]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 648.500116] env[68241]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 648.500116] env[68241]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 648.500116] env[68241]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 648.500116] env[68241]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 648.500116] env[68241]: ERROR neutron response = self.get(path=path, params=params) [ 648.500697] env[68241]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 648.500697] env[68241]: ERROR neutron raise RuntimeError(msg, last_err) [ 648.500697] env[68241]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 648.500697] env[68241]: ERROR neutron [ 651.316156] env[69004]: INFO neutron.common.config [-] Logging enabled! [ 651.316460] env[69004]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 651.316693] env[69004]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69004) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 651.316951] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 651.317049] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 651.318592] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 651.347580] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Session token - acquiring [ 651.348407] env[69004]: DEBUG networking_nsxv3.common.locking [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69004) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 651.348622] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 651.348749] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 651.349191] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 651.360340] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 651.360665] env[69004]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 651.360842] env[69004]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69004) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 653.367579] env[69004]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Session token - acquiring [ 653.367906] env[69004]: DEBUG networking_nsxv3.common.locking [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69004) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 653.368088] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 653.368214] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 653.368768] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 653.379943] env[69004]: DEBUG oslo_concurrency.lockutils [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69004) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 653.380230] env[69004]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 653.380371] env[69004]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69004) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 655.388047] env[69004]: CRITICAL neutron [None req-a0988525-e980-4492-a015-557f14eedd9d None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 655.388047] env[69004]: ERROR neutron Traceback (most recent call last): [ 655.388047] env[69004]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 655.388047] env[69004]: ERROR neutron sys.exit(main()) [ 655.388047] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 655.388047] env[69004]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 655.388047] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 655.388047] env[69004]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 655.388047] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 655.388047] env[69004]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 655.388047] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 655.388047] env[69004]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 655.388047] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 655.388047] env[69004]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 655.388047] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 655.388047] env[69004]: ERROR neutron response = self.get(path=path, params=params) [ 655.388930] env[69004]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 655.388930] env[69004]: ERROR neutron raise RuntimeError(msg, last_err) [ 655.388930] env[69004]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 655.388930] env[69004]: ERROR neutron [ 658.047049] env[69522]: INFO neutron.common.config [-] Logging enabled! [ 658.047322] env[69522]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 658.047943] env[69522]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69522) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 658.047943] env[69522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 658.047943] env[69522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 658.048896] env[69522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 658.063288] env[69522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Session token - acquiring [ 658.064136] env[69522]: DEBUG networking_nsxv3.common.locking [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69522) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 658.064356] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 658.064476] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 658.064841] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 658.074573] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 658.074933] env[69522]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 658.075225] env[69522]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69522) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 660.081935] env[69522]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Session token - acquiring [ 660.082301] env[69522]: DEBUG networking_nsxv3.common.locking [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69522) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 660.082497] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 660.082623] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 660.082934] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 660.092567] env[69522]: DEBUG oslo_concurrency.lockutils [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69522) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 660.092903] env[69522]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 660.093084] env[69522]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69522) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 662.096013] env[69522]: CRITICAL neutron [None req-884b0663-e1fd-47ff-a7cd-2450d84ca891 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 662.096013] env[69522]: ERROR neutron Traceback (most recent call last): [ 662.096013] env[69522]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 662.096013] env[69522]: ERROR neutron sys.exit(main()) [ 662.096013] env[69522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 662.096013] env[69522]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 662.096013] env[69522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 662.096013] env[69522]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 662.096013] env[69522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 662.096013] env[69522]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 662.096013] env[69522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 662.096013] env[69522]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 662.096013] env[69522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 662.096013] env[69522]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 662.096013] env[69522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 662.096013] env[69522]: ERROR neutron response = self.get(path=path, params=params) [ 662.096607] env[69522]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 662.096607] env[69522]: ERROR neutron raise RuntimeError(msg, last_err) [ 662.096607] env[69522]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 662.096607] env[69522]: ERROR neutron [ 664.971069] env[69824]: INFO neutron.common.config [-] Logging enabled! [ 664.971297] env[69824]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 664.971584] env[69824]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=69824) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 664.971824] env[69824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 664.971918] env[69824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 664.972855] env[69824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 664.990354] env[69824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Session token - acquiring [ 664.994199] env[69824]: DEBUG networking_nsxv3.common.locking [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69824) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 664.994199] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 664.994300] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 664.994300] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 665.001661] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 665.001998] env[69824]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 665.002194] env[69824]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=69824) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 667.012155] env[69824]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Session token - acquiring [ 667.012155] env[69824]: DEBUG networking_nsxv3.common.locking [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=69824) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 667.023043] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 667.023043] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 667.023043] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 667.026155] env[69824]: DEBUG oslo_concurrency.lockutils [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=69824) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 667.026155] env[69824]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 667.026155] env[69824]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=69824) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 669.030734] env[69824]: CRITICAL neutron [None req-f7f1b467-157b-482b-8807-b2f9a1bfbe4c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 669.030734] env[69824]: ERROR neutron Traceback (most recent call last): [ 669.030734] env[69824]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 669.030734] env[69824]: ERROR neutron sys.exit(main()) [ 669.030734] env[69824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 669.030734] env[69824]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 669.030734] env[69824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 669.030734] env[69824]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 669.030734] env[69824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 669.030734] env[69824]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 669.030734] env[69824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 669.030734] env[69824]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 669.030734] env[69824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 669.030734] env[69824]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 669.030734] env[69824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 669.030734] env[69824]: ERROR neutron response = self.get(path=path, params=params) [ 669.031351] env[69824]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 669.031351] env[69824]: ERROR neutron raise RuntimeError(msg, last_err) [ 669.031351] env[69824]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 669.031351] env[69824]: ERROR neutron [ 671.991144] env[70106]: INFO neutron.common.config [-] Logging enabled! [ 671.991441] env[70106]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 671.993227] env[70106]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70106) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 671.993227] env[70106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 671.993227] env[70106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 671.993809] env[70106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 672.009158] env[70106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Session token - acquiring [ 672.010001] env[70106]: DEBUG networking_nsxv3.common.locking [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70106) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 672.010219] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 672.010348] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 672.010718] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 672.020761] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 672.021031] env[70106]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 672.021183] env[70106]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70106) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 674.029074] env[70106]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Session token - acquiring [ 674.029453] env[70106]: DEBUG networking_nsxv3.common.locking [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70106) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 674.029561] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 674.029684] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 674.030025] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 674.039679] env[70106]: DEBUG oslo_concurrency.lockutils [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70106) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 674.040047] env[70106]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 674.040224] env[70106]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70106) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 676.043539] env[70106]: CRITICAL neutron [None req-9bdd9b9d-7226-4d9f-9134-d1b22f331ee8 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 676.043539] env[70106]: ERROR neutron Traceback (most recent call last): [ 676.043539] env[70106]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 676.043539] env[70106]: ERROR neutron sys.exit(main()) [ 676.043539] env[70106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 676.043539] env[70106]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 676.043539] env[70106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 676.043539] env[70106]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 676.043539] env[70106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 676.043539] env[70106]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 676.043539] env[70106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 676.043539] env[70106]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 676.043539] env[70106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 676.043539] env[70106]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 676.043539] env[70106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 676.043539] env[70106]: ERROR neutron response = self.get(path=path, params=params) [ 676.044293] env[70106]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 676.044293] env[70106]: ERROR neutron raise RuntimeError(msg, last_err) [ 676.044293] env[70106]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 676.044293] env[70106]: ERROR neutron [ 678.934708] env[70374]: INFO neutron.common.config [-] Logging enabled! [ 678.935230] env[70374]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 678.935293] env[70374]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70374) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 678.935497] env[70374]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 678.935651] env[70374]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 678.936947] env[70374]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 678.950805] env[70374]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Session token - acquiring [ 678.951753] env[70374]: DEBUG networking_nsxv3.common.locking [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70374) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 678.951882] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 678.952025] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 678.952406] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 678.976760] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 678.977147] env[70374]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 678.977389] env[70374]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70374) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 680.985324] env[70374]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Session token - acquiring [ 680.985635] env[70374]: DEBUG networking_nsxv3.common.locking [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70374) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 680.985826] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 680.985947] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 680.986227] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 680.996348] env[70374]: DEBUG oslo_concurrency.lockutils [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70374) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 680.996603] env[70374]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 680.996742] env[70374]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70374) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 682.999689] env[70374]: CRITICAL neutron [None req-d0926513-1efe-4324-982c-fe39da078505 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 682.999689] env[70374]: ERROR neutron Traceback (most recent call last): [ 682.999689] env[70374]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 682.999689] env[70374]: ERROR neutron sys.exit(main()) [ 682.999689] env[70374]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 682.999689] env[70374]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 682.999689] env[70374]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 682.999689] env[70374]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 682.999689] env[70374]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 682.999689] env[70374]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 682.999689] env[70374]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 682.999689] env[70374]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 682.999689] env[70374]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 682.999689] env[70374]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 682.999689] env[70374]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 682.999689] env[70374]: ERROR neutron response = self.get(path=path, params=params) [ 683.000480] env[70374]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 683.000480] env[70374]: ERROR neutron raise RuntimeError(msg, last_err) [ 683.000480] env[70374]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 683.000480] env[70374]: ERROR neutron [ 685.723452] env[70966]: INFO neutron.common.config [-] Logging enabled! [ 685.723452] env[70966]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 685.723821] env[70966]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=70966) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 685.724052] env[70966]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 685.724173] env[70966]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 685.725026] env[70966]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 685.738995] env[70966]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Session token - acquiring [ 685.739866] env[70966]: DEBUG networking_nsxv3.common.locking [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70966) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 685.740135] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 685.740269] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 685.740638] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 685.750400] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 685.750662] env[70966]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 685.750815] env[70966]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=70966) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 687.757773] env[70966]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Session token - acquiring [ 687.758072] env[70966]: DEBUG networking_nsxv3.common.locking [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=70966) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 687.758255] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 687.758389] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 687.758776] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 687.769421] env[70966]: DEBUG oslo_concurrency.lockutils [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=70966) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 687.769788] env[70966]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 687.769949] env[70966]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=70966) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 689.772840] env[70966]: CRITICAL neutron [None req-29ce81b3-cc5e-4d69-b6c3-1f74bf5147eb None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 689.772840] env[70966]: ERROR neutron Traceback (most recent call last): [ 689.772840] env[70966]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 689.772840] env[70966]: ERROR neutron sys.exit(main()) [ 689.772840] env[70966]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 689.772840] env[70966]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 689.772840] env[70966]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 689.772840] env[70966]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 689.772840] env[70966]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 689.772840] env[70966]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 689.772840] env[70966]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 689.772840] env[70966]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 689.772840] env[70966]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 689.772840] env[70966]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 689.772840] env[70966]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 689.772840] env[70966]: ERROR neutron response = self.get(path=path, params=params) [ 689.773559] env[70966]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 689.773559] env[70966]: ERROR neutron raise RuntimeError(msg, last_err) [ 689.773559] env[70966]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 689.773559] env[70966]: ERROR neutron [ 692.495958] env[71360]: INFO neutron.common.config [-] Logging enabled! [ 692.496211] env[71360]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 692.496609] env[71360]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71360) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 692.496860] env[71360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 692.496944] env[71360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 692.497902] env[71360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 692.511865] env[71360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Session token - acquiring [ 692.512731] env[71360]: DEBUG networking_nsxv3.common.locking [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71360) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 692.512945] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 692.513179] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 692.513541] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 692.525232] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 692.525482] env[71360]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 692.525624] env[71360]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71360) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 694.533388] env[71360]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Session token - acquiring [ 694.533782] env[71360]: DEBUG networking_nsxv3.common.locking [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71360) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 694.534030] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 694.534207] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 694.534550] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 694.544610] env[71360]: DEBUG oslo_concurrency.lockutils [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71360) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 694.544987] env[71360]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 694.545193] env[71360]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71360) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 696.548596] env[71360]: CRITICAL neutron [None req-967f2eb6-97b4-4003-8a1d-cc298dbf9a0b None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 696.548596] env[71360]: ERROR neutron Traceback (most recent call last): [ 696.548596] env[71360]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 696.548596] env[71360]: ERROR neutron sys.exit(main()) [ 696.548596] env[71360]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 696.548596] env[71360]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 696.548596] env[71360]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 696.548596] env[71360]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 696.548596] env[71360]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 696.548596] env[71360]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 696.548596] env[71360]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 696.548596] env[71360]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 696.548596] env[71360]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 696.548596] env[71360]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 696.548596] env[71360]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 696.548596] env[71360]: ERROR neutron response = self.get(path=path, params=params) [ 696.549222] env[71360]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 696.549222] env[71360]: ERROR neutron raise RuntimeError(msg, last_err) [ 696.549222] env[71360]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 696.549222] env[71360]: ERROR neutron [ 699.311599] env[71733]: INFO neutron.common.config [-] Logging enabled! [ 699.311599] env[71733]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 699.311997] env[71733]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=71733) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 699.312217] env[71733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 699.312309] env[71733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 699.313212] env[71733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 699.327631] env[71733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Session token - acquiring [ 699.328526] env[71733]: DEBUG networking_nsxv3.common.locking [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71733) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 699.328778] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 699.328996] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 699.329319] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 699.340154] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 699.340417] env[71733]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 699.340554] env[71733]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=71733) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 701.348569] env[71733]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Session token - acquiring [ 701.348939] env[71733]: DEBUG networking_nsxv3.common.locking [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=71733) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 701.349031] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 701.349155] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 701.349445] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 701.359184] env[71733]: DEBUG oslo_concurrency.lockutils [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=71733) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 701.359486] env[71733]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 701.359647] env[71733]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=71733) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 703.361435] env[71733]: CRITICAL neutron [None req-631bf502-936c-4fec-9d4a-75df8eab2151 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 703.361435] env[71733]: ERROR neutron Traceback (most recent call last): [ 703.361435] env[71733]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 703.361435] env[71733]: ERROR neutron sys.exit(main()) [ 703.361435] env[71733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 703.361435] env[71733]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 703.361435] env[71733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 703.361435] env[71733]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 703.361435] env[71733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 703.361435] env[71733]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 703.361435] env[71733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 703.361435] env[71733]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 703.361435] env[71733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 703.361435] env[71733]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 703.361435] env[71733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 703.361435] env[71733]: ERROR neutron response = self.get(path=path, params=params) [ 703.362059] env[71733]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 703.362059] env[71733]: ERROR neutron raise RuntimeError(msg, last_err) [ 703.362059] env[71733]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 703.362059] env[71733]: ERROR neutron [ 706.195520] env[72158]: INFO neutron.common.config [-] Logging enabled! [ 706.195800] env[72158]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 706.196039] env[72158]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72158) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 706.196275] env[72158]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 706.196363] env[72158]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 706.197264] env[72158]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 706.212398] env[72158]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Session token - acquiring [ 706.213227] env[72158]: DEBUG networking_nsxv3.common.locking [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72158) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 706.213455] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 706.213578] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 706.213963] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 706.224466] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 706.224880] env[72158]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 706.225076] env[72158]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72158) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 708.233163] env[72158]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Session token - acquiring [ 708.233549] env[72158]: DEBUG networking_nsxv3.common.locking [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72158) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 708.233721] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 708.233786] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 708.234079] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 708.244469] env[72158]: DEBUG oslo_concurrency.lockutils [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72158) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 708.244757] env[72158]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 708.244904] env[72158]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72158) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 710.247794] env[72158]: CRITICAL neutron [None req-e68042cd-4ea7-4e8a-8c47-ae9b58dbe7f1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 710.247794] env[72158]: ERROR neutron Traceback (most recent call last): [ 710.247794] env[72158]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 710.247794] env[72158]: ERROR neutron sys.exit(main()) [ 710.247794] env[72158]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 710.247794] env[72158]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 710.247794] env[72158]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 710.247794] env[72158]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 710.247794] env[72158]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 710.247794] env[72158]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 710.247794] env[72158]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 710.247794] env[72158]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 710.247794] env[72158]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 710.247794] env[72158]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 710.247794] env[72158]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 710.247794] env[72158]: ERROR neutron response = self.get(path=path, params=params) [ 710.248491] env[72158]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 710.248491] env[72158]: ERROR neutron raise RuntimeError(msg, last_err) [ 710.248491] env[72158]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 710.248491] env[72158]: ERROR neutron [ 713.045157] env[72734]: INFO neutron.common.config [-] Logging enabled! [ 713.045157] env[72734]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 713.045433] env[72734]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=72734) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 713.047192] env[72734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 713.047192] env[72734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 713.047192] env[72734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 713.064365] env[72734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Session token - acquiring [ 713.065240] env[72734]: DEBUG networking_nsxv3.common.locking [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72734) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 713.065559] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 713.065559] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 713.065932] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 713.077990] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 713.078572] env[72734]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 713.078725] env[72734]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=72734) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 715.085799] env[72734]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Session token - acquiring [ 715.086171] env[72734]: DEBUG networking_nsxv3.common.locking [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=72734) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 715.086294] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 715.086427] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 715.086808] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 715.096940] env[72734]: DEBUG oslo_concurrency.lockutils [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=72734) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 715.097268] env[72734]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 715.097483] env[72734]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=72734) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 717.100398] env[72734]: CRITICAL neutron [None req-76a714b4-23c4-4d19-9319-7c1c9e65f8c3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 717.100398] env[72734]: ERROR neutron Traceback (most recent call last): [ 717.100398] env[72734]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 717.100398] env[72734]: ERROR neutron sys.exit(main()) [ 717.100398] env[72734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 717.100398] env[72734]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 717.100398] env[72734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 717.100398] env[72734]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 717.100398] env[72734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 717.100398] env[72734]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 717.100398] env[72734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 717.100398] env[72734]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 717.100398] env[72734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 717.100398] env[72734]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 717.100398] env[72734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 717.100398] env[72734]: ERROR neutron response = self.get(path=path, params=params) [ 717.101588] env[72734]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 717.101588] env[72734]: ERROR neutron raise RuntimeError(msg, last_err) [ 717.101588] env[72734]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 717.101588] env[72734]: ERROR neutron [ 719.859251] env[73127]: INFO neutron.common.config [-] Logging enabled! [ 719.859529] env[73127]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 719.859730] env[73127]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73127) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 719.859976] env[73127]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 719.860085] env[73127]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 719.860967] env[73127]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 719.874498] env[73127]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Session token - acquiring [ 719.875442] env[73127]: DEBUG networking_nsxv3.common.locking [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73127) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 719.875639] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 719.875762] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 719.876170] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 719.885616] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 719.885856] env[73127]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 719.885997] env[73127]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73127) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 721.899936] env[73127]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Session token - acquiring [ 721.900282] env[73127]: DEBUG networking_nsxv3.common.locking [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73127) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 721.900517] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 721.900562] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 721.900833] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 721.914021] env[73127]: DEBUG oslo_concurrency.lockutils [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73127) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 721.914312] env[73127]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 721.914487] env[73127]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73127) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 723.920209] env[73127]: CRITICAL neutron [None req-b53272bb-7b35-4013-a01a-2f494b60c1cd None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 723.920209] env[73127]: ERROR neutron Traceback (most recent call last): [ 723.920209] env[73127]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 723.920209] env[73127]: ERROR neutron sys.exit(main()) [ 723.920209] env[73127]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 723.920209] env[73127]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 723.920209] env[73127]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 723.920209] env[73127]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 723.920209] env[73127]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 723.920209] env[73127]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 723.920209] env[73127]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 723.920209] env[73127]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 723.920209] env[73127]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 723.920209] env[73127]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 723.920209] env[73127]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 723.920209] env[73127]: ERROR neutron response = self.get(path=path, params=params) [ 723.920916] env[73127]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 723.920916] env[73127]: ERROR neutron raise RuntimeError(msg, last_err) [ 723.920916] env[73127]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 723.920916] env[73127]: ERROR neutron [ 726.574199] env[73622]: INFO neutron.common.config [-] Logging enabled! [ 726.574449] env[73622]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 726.574724] env[73622]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=73622) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 726.574972] env[73622]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 726.575070] env[73622]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 726.575969] env[73622]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 726.590813] env[73622]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Session token - acquiring [ 726.591815] env[73622]: DEBUG networking_nsxv3.common.locking [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73622) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 726.591950] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 726.592056] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 726.592586] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 726.602002] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 726.602595] env[73622]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 726.602808] env[73622]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=73622) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 728.609853] env[73622]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Session token - acquiring [ 728.610121] env[73622]: DEBUG networking_nsxv3.common.locking [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=73622) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 728.610298] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 728.610427] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 728.610702] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 728.620330] env[73622]: DEBUG oslo_concurrency.lockutils [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=73622) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 728.620674] env[73622]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 728.620835] env[73622]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=73622) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 730.623719] env[73622]: CRITICAL neutron [None req-d467ad3e-9c45-4403-b172-32dfcade19e6 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 730.623719] env[73622]: ERROR neutron Traceback (most recent call last): [ 730.623719] env[73622]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 730.623719] env[73622]: ERROR neutron sys.exit(main()) [ 730.623719] env[73622]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 730.623719] env[73622]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 730.623719] env[73622]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 730.623719] env[73622]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 730.623719] env[73622]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 730.623719] env[73622]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 730.623719] env[73622]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 730.623719] env[73622]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 730.623719] env[73622]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 730.623719] env[73622]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 730.623719] env[73622]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 730.623719] env[73622]: ERROR neutron response = self.get(path=path, params=params) [ 730.624540] env[73622]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 730.624540] env[73622]: ERROR neutron raise RuntimeError(msg, last_err) [ 730.624540] env[73622]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 730.624540] env[73622]: ERROR neutron [ 733.424343] env[74054]: INFO neutron.common.config [-] Logging enabled! [ 733.424611] env[74054]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 733.424823] env[74054]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74054) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 733.425054] env[74054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 733.425144] env[74054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 733.426060] env[74054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 733.440117] env[74054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Session token - acquiring [ 733.440947] env[74054]: DEBUG networking_nsxv3.common.locking [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74054) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 733.441156] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 733.441282] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 733.441661] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 733.452147] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 733.452147] env[74054]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 733.452147] env[74054]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74054) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 735.459960] env[74054]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Session token - acquiring [ 735.460335] env[74054]: DEBUG networking_nsxv3.common.locking [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74054) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 735.460625] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 735.460625] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 735.460874] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 735.470354] env[74054]: DEBUG oslo_concurrency.lockutils [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74054) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 735.470643] env[74054]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 735.470824] env[74054]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74054) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 737.473193] env[74054]: CRITICAL neutron [None req-00069bc0-0c2a-4d01-b38e-73813d4cc208 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 737.473193] env[74054]: ERROR neutron Traceback (most recent call last): [ 737.473193] env[74054]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 737.473193] env[74054]: ERROR neutron sys.exit(main()) [ 737.473193] env[74054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 737.473193] env[74054]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 737.473193] env[74054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 737.473193] env[74054]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 737.473193] env[74054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 737.473193] env[74054]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 737.473193] env[74054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 737.473193] env[74054]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 737.473193] env[74054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 737.473193] env[74054]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 737.473193] env[74054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 737.473193] env[74054]: ERROR neutron response = self.get(path=path, params=params) [ 737.473912] env[74054]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 737.473912] env[74054]: ERROR neutron raise RuntimeError(msg, last_err) [ 737.473912] env[74054]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 737.473912] env[74054]: ERROR neutron [ 740.210947] env[74581]: INFO neutron.common.config [-] Logging enabled! [ 740.211226] env[74581]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 740.211570] env[74581]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=74581) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 740.211894] env[74581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 740.211992] env[74581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 740.212915] env[74581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 740.233342] env[74581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Session token - acquiring [ 740.233485] env[74581]: DEBUG networking_nsxv3.common.locking [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74581) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 740.237020] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 740.237192] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 740.237631] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 740.248150] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 740.248328] env[74581]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 740.248532] env[74581]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=74581) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 742.258405] env[74581]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Session token - acquiring [ 742.259067] env[74581]: DEBUG networking_nsxv3.common.locking [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=74581) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 742.259067] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 742.259213] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 742.259300] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 742.269345] env[74581]: DEBUG oslo_concurrency.lockutils [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=74581) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 742.270001] env[74581]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 742.270190] env[74581]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=74581) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 744.272865] env[74581]: CRITICAL neutron [None req-ad14d654-7a15-4974-a7fc-9b34e89327e7 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 744.272865] env[74581]: ERROR neutron Traceback (most recent call last): [ 744.272865] env[74581]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 744.272865] env[74581]: ERROR neutron sys.exit(main()) [ 744.272865] env[74581]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 744.272865] env[74581]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 744.272865] env[74581]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 744.272865] env[74581]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 744.272865] env[74581]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 744.272865] env[74581]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 744.272865] env[74581]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 744.272865] env[74581]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 744.272865] env[74581]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 744.272865] env[74581]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 744.272865] env[74581]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 744.272865] env[74581]: ERROR neutron response = self.get(path=path, params=params) [ 744.273588] env[74581]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 744.273588] env[74581]: ERROR neutron raise RuntimeError(msg, last_err) [ 744.273588] env[74581]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 744.273588] env[74581]: ERROR neutron [ 746.931735] env[75122]: INFO neutron.common.config [-] Logging enabled! [ 746.932052] env[75122]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 746.932271] env[75122]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75122) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 746.932505] env[75122]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 746.932597] env[75122]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 746.933484] env[75122]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 746.947276] env[75122]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Session token - acquiring [ 746.948141] env[75122]: DEBUG networking_nsxv3.common.locking [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75122) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 746.948360] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 746.949068] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 746.949068] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 746.958235] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 746.958665] env[75122]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 746.958665] env[75122]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75122) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 748.965944] env[75122]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Session token - acquiring [ 748.966278] env[75122]: DEBUG networking_nsxv3.common.locking [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75122) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 748.966477] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 748.966610] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 748.966891] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 748.978240] env[75122]: DEBUG oslo_concurrency.lockutils [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75122) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 748.978920] env[75122]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 748.979126] env[75122]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75122) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 750.980882] env[75122]: CRITICAL neutron [None req-784918e3-4eb6-457a-ad32-583be25d2d81 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 750.980882] env[75122]: ERROR neutron Traceback (most recent call last): [ 750.980882] env[75122]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 750.980882] env[75122]: ERROR neutron sys.exit(main()) [ 750.980882] env[75122]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 750.980882] env[75122]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 750.980882] env[75122]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 750.980882] env[75122]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 750.980882] env[75122]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 750.980882] env[75122]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 750.980882] env[75122]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 750.980882] env[75122]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 750.980882] env[75122]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 750.980882] env[75122]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 750.980882] env[75122]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 750.980882] env[75122]: ERROR neutron response = self.get(path=path, params=params) [ 750.981623] env[75122]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 750.981623] env[75122]: ERROR neutron raise RuntimeError(msg, last_err) [ 750.981623] env[75122]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 750.981623] env[75122]: ERROR neutron [ 753.569293] env[75550]: INFO neutron.common.config [-] Logging enabled! [ 753.569660] env[75550]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 753.569833] env[75550]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75550) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 753.570084] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 753.570179] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 753.571047] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 753.587135] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Session token - acquiring [ 753.587135] env[75550]: DEBUG networking_nsxv3.common.locking [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75550) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 753.587332] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 753.587332] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 753.587332] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 753.595494] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 753.595783] env[75550]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 753.595932] env[75550]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75550) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 755.603499] env[75550]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Session token - acquiring [ 755.603499] env[75550]: DEBUG networking_nsxv3.common.locking [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75550) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 755.603889] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 755.603889] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 755.603889] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 755.612698] env[75550]: DEBUG oslo_concurrency.lockutils [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75550) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 755.613191] env[75550]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 755.613481] env[75550]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75550) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 757.617053] env[75550]: CRITICAL neutron [None req-69d08b72-8745-4c9a-8222-238756a73494 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 757.617053] env[75550]: ERROR neutron Traceback (most recent call last): [ 757.617053] env[75550]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 757.617053] env[75550]: ERROR neutron sys.exit(main()) [ 757.617053] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 757.617053] env[75550]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 757.617053] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 757.617053] env[75550]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 757.617053] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 757.617053] env[75550]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 757.617053] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 757.617053] env[75550]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 757.617053] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 757.617053] env[75550]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 757.617053] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 757.617053] env[75550]: ERROR neutron response = self.get(path=path, params=params) [ 757.618170] env[75550]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 757.618170] env[75550]: ERROR neutron raise RuntimeError(msg, last_err) [ 757.618170] env[75550]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 757.618170] env[75550]: ERROR neutron [ 760.175830] env[75728]: INFO neutron.common.config [-] Logging enabled! [ 760.176339] env[75728]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 760.176339] env[75728]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75728) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 760.176576] env[75728]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 760.176672] env[75728]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 760.177545] env[75728]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 760.190614] env[75728]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Session token - acquiring [ 760.191340] env[75728]: DEBUG networking_nsxv3.common.locking [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75728) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 760.191539] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 760.191671] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 760.192052] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 760.201526] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 760.201783] env[75728]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 760.202013] env[75728]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75728) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 762.209863] env[75728]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Session token - acquiring [ 762.210208] env[75728]: DEBUG networking_nsxv3.common.locking [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75728) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 762.210349] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 762.210475] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 762.210763] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 762.220635] env[75728]: DEBUG oslo_concurrency.lockutils [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75728) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 762.220972] env[75728]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 762.221178] env[75728]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75728) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 764.224091] env[75728]: CRITICAL neutron [None req-2b15a14b-0af3-4d78-b044-128a400d0d44 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 764.224091] env[75728]: ERROR neutron Traceback (most recent call last): [ 764.224091] env[75728]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 764.224091] env[75728]: ERROR neutron sys.exit(main()) [ 764.224091] env[75728]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 764.224091] env[75728]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 764.224091] env[75728]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 764.224091] env[75728]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 764.224091] env[75728]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 764.224091] env[75728]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 764.224091] env[75728]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 764.224091] env[75728]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 764.224091] env[75728]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 764.224091] env[75728]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 764.224091] env[75728]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 764.224091] env[75728]: ERROR neutron response = self.get(path=path, params=params) [ 764.224791] env[75728]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 764.224791] env[75728]: ERROR neutron raise RuntimeError(msg, last_err) [ 764.224791] env[75728]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 764.224791] env[75728]: ERROR neutron [ 766.702934] env[75778]: INFO neutron.common.config [-] Logging enabled! [ 766.703367] env[75778]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 766.703411] env[75778]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75778) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 766.703620] env[75778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 766.703712] env[75778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 766.704611] env[75778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 766.717886] env[75778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Session token - acquiring [ 766.718616] env[75778]: DEBUG networking_nsxv3.common.locking [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75778) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 766.718815] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 766.718942] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 766.719291] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 766.728415] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 766.728640] env[75778]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 766.728822] env[75778]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75778) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 768.736061] env[75778]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Session token - acquiring [ 768.736527] env[75778]: DEBUG networking_nsxv3.common.locking [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75778) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 768.736527] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 768.736690] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 768.736912] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 768.746600] env[75778]: DEBUG oslo_concurrency.lockutils [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75778) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 768.746972] env[75778]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 768.747132] env[75778]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75778) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 770.748911] env[75778]: CRITICAL neutron [None req-5e354438-c4a1-4eb2-8ba8-382e2481818c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 770.748911] env[75778]: ERROR neutron Traceback (most recent call last): [ 770.748911] env[75778]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 770.748911] env[75778]: ERROR neutron sys.exit(main()) [ 770.748911] env[75778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 770.748911] env[75778]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 770.748911] env[75778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 770.748911] env[75778]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 770.748911] env[75778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 770.748911] env[75778]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 770.748911] env[75778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 770.748911] env[75778]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 770.748911] env[75778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 770.748911] env[75778]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 770.748911] env[75778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 770.748911] env[75778]: ERROR neutron response = self.get(path=path, params=params) [ 770.749756] env[75778]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 770.749756] env[75778]: ERROR neutron raise RuntimeError(msg, last_err) [ 770.749756] env[75778]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 770.749756] env[75778]: ERROR neutron [ 773.262351] env[75782]: INFO neutron.common.config [-] Logging enabled! [ 773.262808] env[75782]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 773.262808] env[75782]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75782) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 773.263040] env[75782]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 773.263135] env[75782]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 773.263964] env[75782]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 773.277324] env[75782]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Session token - acquiring [ 773.278042] env[75782]: DEBUG networking_nsxv3.common.locking [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75782) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 773.278240] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 773.278368] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 773.278740] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 773.287553] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 773.287798] env[75782]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 773.287953] env[75782]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75782) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 775.293967] env[75782]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Session token - acquiring [ 775.294345] env[75782]: DEBUG networking_nsxv3.common.locking [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75782) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 775.294443] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 775.294570] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 775.294848] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 775.309594] env[75782]: DEBUG oslo_concurrency.lockutils [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75782) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 775.310051] env[75782]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 775.310284] env[75782]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75782) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 777.312906] env[75782]: CRITICAL neutron [None req-7f6f2dc6-4977-4360-9ce9-5a5bc2d12903 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 777.312906] env[75782]: ERROR neutron Traceback (most recent call last): [ 777.312906] env[75782]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 777.312906] env[75782]: ERROR neutron sys.exit(main()) [ 777.312906] env[75782]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 777.312906] env[75782]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 777.312906] env[75782]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 777.312906] env[75782]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 777.312906] env[75782]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 777.312906] env[75782]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 777.312906] env[75782]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 777.312906] env[75782]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 777.312906] env[75782]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 777.312906] env[75782]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 777.312906] env[75782]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 777.312906] env[75782]: ERROR neutron response = self.get(path=path, params=params) [ 777.313722] env[75782]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 777.313722] env[75782]: ERROR neutron raise RuntimeError(msg, last_err) [ 777.313722] env[75782]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 777.313722] env[75782]: ERROR neutron [ 779.654190] env[75788]: INFO neutron.common.config [-] Logging enabled! [ 779.654610] env[75788]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 779.654737] env[75788]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75788) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 779.654904] env[75788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 779.654999] env[75788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 779.655876] env[75788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 779.669617] env[75788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Session token - acquiring [ 779.670219] env[75788]: DEBUG networking_nsxv3.common.locking [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75788) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 779.670416] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 779.670544] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 779.670892] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 779.679863] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 779.680109] env[75788]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 779.680294] env[75788]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75788) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 781.688923] env[75788]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Session token - acquiring [ 781.688923] env[75788]: DEBUG networking_nsxv3.common.locking [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75788) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 781.689394] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 781.689394] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 781.689394] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 781.698505] env[75788]: DEBUG oslo_concurrency.lockutils [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75788) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 781.698780] env[75788]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 781.698931] env[75788]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75788) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 783.701939] env[75788]: CRITICAL neutron [None req-8f10f7b5-08aa-4385-aef1-8682cfd35115 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 783.701939] env[75788]: ERROR neutron Traceback (most recent call last): [ 783.701939] env[75788]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 783.701939] env[75788]: ERROR neutron sys.exit(main()) [ 783.701939] env[75788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 783.701939] env[75788]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 783.701939] env[75788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 783.701939] env[75788]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 783.701939] env[75788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 783.701939] env[75788]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 783.701939] env[75788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 783.701939] env[75788]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 783.701939] env[75788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 783.701939] env[75788]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 783.701939] env[75788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 783.701939] env[75788]: ERROR neutron response = self.get(path=path, params=params) [ 783.702776] env[75788]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 783.702776] env[75788]: ERROR neutron raise RuntimeError(msg, last_err) [ 783.702776] env[75788]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 783.702776] env[75788]: ERROR neutron [ 786.164503] env[75794]: INFO neutron.common.config [-] Logging enabled! [ 786.164923] env[75794]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 786.165045] env[75794]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75794) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 786.165244] env[75794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 786.165344] env[75794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 786.166207] env[75794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 786.179568] env[75794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Session token - acquiring [ 786.180445] env[75794]: DEBUG networking_nsxv3.common.locking [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75794) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 786.180651] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 786.180779] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 786.181133] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 786.190343] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 786.190578] env[75794]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 786.190831] env[75794]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75794) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 788.199181] env[75794]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Session token - acquiring [ 788.199542] env[75794]: DEBUG networking_nsxv3.common.locking [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75794) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 788.199662] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 788.199811] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 788.200101] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 788.209589] env[75794]: DEBUG oslo_concurrency.lockutils [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75794) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 788.209916] env[75794]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 788.210095] env[75794]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75794) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 790.212920] env[75794]: CRITICAL neutron [None req-a8a7e2e8-141a-4998-a8f7-ea73eb8c9d91 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 790.212920] env[75794]: ERROR neutron Traceback (most recent call last): [ 790.212920] env[75794]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 790.212920] env[75794]: ERROR neutron sys.exit(main()) [ 790.212920] env[75794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 790.212920] env[75794]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 790.212920] env[75794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 790.212920] env[75794]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 790.212920] env[75794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 790.212920] env[75794]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 790.212920] env[75794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 790.212920] env[75794]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 790.212920] env[75794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 790.212920] env[75794]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 790.212920] env[75794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 790.212920] env[75794]: ERROR neutron response = self.get(path=path, params=params) [ 790.213689] env[75794]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 790.213689] env[75794]: ERROR neutron raise RuntimeError(msg, last_err) [ 790.213689] env[75794]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 790.213689] env[75794]: ERROR neutron [ 792.671975] env[75798]: INFO neutron.common.config [-] Logging enabled! [ 792.672351] env[75798]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 792.672460] env[75798]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75798) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 792.672692] env[75798]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 792.672790] env[75798]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 792.673635] env[75798]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 792.689323] env[75798]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Session token - acquiring [ 792.689323] env[75798]: DEBUG networking_nsxv3.common.locking [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75798) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 792.689517] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 792.689517] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 792.689517] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 792.697361] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 792.697758] env[75798]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 792.698058] env[75798]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75798) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 794.705883] env[75798]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Session token - acquiring [ 794.706715] env[75798]: DEBUG networking_nsxv3.common.locking [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75798) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 794.707130] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 794.707397] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 794.707838] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 794.717284] env[75798]: DEBUG oslo_concurrency.lockutils [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75798) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 794.717718] env[75798]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 794.718047] env[75798]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75798) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 796.721096] env[75798]: CRITICAL neutron [None req-d2031076-dd32-47c7-bbfe-6c3c459564a1 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 796.721096] env[75798]: ERROR neutron Traceback (most recent call last): [ 796.721096] env[75798]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 796.721096] env[75798]: ERROR neutron sys.exit(main()) [ 796.721096] env[75798]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 796.721096] env[75798]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 796.721096] env[75798]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 796.721096] env[75798]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 796.721096] env[75798]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 796.721096] env[75798]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 796.721096] env[75798]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 796.721096] env[75798]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 796.721096] env[75798]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 796.721096] env[75798]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 796.721096] env[75798]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 796.721096] env[75798]: ERROR neutron response = self.get(path=path, params=params) [ 796.722242] env[75798]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 796.722242] env[75798]: ERROR neutron raise RuntimeError(msg, last_err) [ 796.722242] env[75798]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 796.722242] env[75798]: ERROR neutron [ 799.198286] env[75846]: INFO neutron.common.config [-] Logging enabled! [ 799.198778] env[75846]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 799.198923] env[75846]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75846) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 799.199084] env[75846]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 799.199187] env[75846]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 799.200214] env[75846]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 799.214137] env[75846]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Session token - acquiring [ 799.214885] env[75846]: DEBUG networking_nsxv3.common.locking [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75846) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 799.215092] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 799.215222] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 799.215577] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 799.225422] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 799.225678] env[75846]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 799.225829] env[75846]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75846) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 801.233756] env[75846]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Session token - acquiring [ 801.234115] env[75846]: DEBUG networking_nsxv3.common.locking [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75846) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 801.234264] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 801.234379] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 801.234669] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 801.244018] env[75846]: DEBUG oslo_concurrency.lockutils [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75846) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 801.244352] env[75846]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 801.244542] env[75846]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75846) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 803.247617] env[75846]: CRITICAL neutron [None req-abd3cc8f-3738-42c2-a762-c1cb5622b16c None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 803.247617] env[75846]: ERROR neutron Traceback (most recent call last): [ 803.247617] env[75846]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 803.247617] env[75846]: ERROR neutron sys.exit(main()) [ 803.247617] env[75846]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 803.247617] env[75846]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 803.247617] env[75846]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 803.247617] env[75846]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 803.247617] env[75846]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 803.247617] env[75846]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 803.247617] env[75846]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 803.247617] env[75846]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 803.247617] env[75846]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 803.247617] env[75846]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 803.247617] env[75846]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 803.247617] env[75846]: ERROR neutron response = self.get(path=path, params=params) [ 803.248520] env[75846]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 803.248520] env[75846]: ERROR neutron raise RuntimeError(msg, last_err) [ 803.248520] env[75846]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 803.248520] env[75846]: ERROR neutron [ 805.666709] env[75853]: INFO neutron.common.config [-] Logging enabled! [ 805.667148] env[75853]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 805.667218] env[75853]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75853) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 805.667405] env[75853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 805.667522] env[75853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 805.668415] env[75853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 805.682067] env[75853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Session token - acquiring [ 805.682968] env[75853]: DEBUG networking_nsxv3.common.locking [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75853) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 805.683164] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 805.683299] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 805.683655] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 805.692508] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 805.692827] env[75853]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 805.693032] env[75853]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75853) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 807.701009] env[75853]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Session token - acquiring [ 807.701325] env[75853]: DEBUG networking_nsxv3.common.locking [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75853) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 807.701525] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 807.701655] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 807.701932] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 807.711389] env[75853]: DEBUG oslo_concurrency.lockutils [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75853) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 807.711703] env[75853]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 807.711822] env[75853]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75853) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 809.714758] env[75853]: CRITICAL neutron [None req-e5724e22-c350-403a-8c52-e5b7e6de2381 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 809.714758] env[75853]: ERROR neutron Traceback (most recent call last): [ 809.714758] env[75853]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 809.714758] env[75853]: ERROR neutron sys.exit(main()) [ 809.714758] env[75853]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 809.714758] env[75853]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 809.714758] env[75853]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 809.714758] env[75853]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 809.714758] env[75853]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 809.714758] env[75853]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 809.714758] env[75853]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 809.714758] env[75853]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 809.714758] env[75853]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 809.714758] env[75853]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 809.714758] env[75853]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 809.714758] env[75853]: ERROR neutron response = self.get(path=path, params=params) [ 809.715572] env[75853]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 809.715572] env[75853]: ERROR neutron raise RuntimeError(msg, last_err) [ 809.715572] env[75853]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 809.715572] env[75853]: ERROR neutron [ 812.156584] env[75857]: INFO neutron.common.config [-] Logging enabled! [ 812.156913] env[75857]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 812.157056] env[75857]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75857) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 812.157278] env[75857]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 812.157374] env[75857]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 812.158249] env[75857]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 812.175350] env[75857]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Session token - acquiring [ 812.176125] env[75857]: DEBUG networking_nsxv3.common.locking [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75857) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 812.176335] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 812.176465] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 812.176816] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 812.186246] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 812.186556] env[75857]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 812.186751] env[75857]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75857) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 814.193792] env[75857]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Session token - acquiring [ 814.194097] env[75857]: DEBUG networking_nsxv3.common.locking [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75857) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 814.194282] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 814.194405] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 814.194688] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 814.204049] env[75857]: DEBUG oslo_concurrency.lockutils [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75857) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 814.204337] env[75857]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 814.204444] env[75857]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75857) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 816.207353] env[75857]: CRITICAL neutron [None req-343f4b4b-4745-4755-a7ab-3dfa5ab493ec None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 816.207353] env[75857]: ERROR neutron Traceback (most recent call last): [ 816.207353] env[75857]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 816.207353] env[75857]: ERROR neutron sys.exit(main()) [ 816.207353] env[75857]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 816.207353] env[75857]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 816.207353] env[75857]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 816.207353] env[75857]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 816.207353] env[75857]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 816.207353] env[75857]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 816.207353] env[75857]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 816.207353] env[75857]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 816.207353] env[75857]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 816.207353] env[75857]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 816.207353] env[75857]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 816.207353] env[75857]: ERROR neutron response = self.get(path=path, params=params) [ 816.208500] env[75857]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 816.208500] env[75857]: ERROR neutron raise RuntimeError(msg, last_err) [ 816.208500] env[75857]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 816.208500] env[75857]: ERROR neutron [ 818.648976] env[75863]: INFO neutron.common.config [-] Logging enabled! [ 818.649393] env[75863]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 818.649471] env[75863]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75863) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 818.649689] env[75863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 818.649787] env[75863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 818.650645] env[75863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 818.664406] env[75863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Session token - acquiring [ 818.665246] env[75863]: DEBUG networking_nsxv3.common.locking [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75863) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 818.665516] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 818.665572] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 818.665917] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 818.675587] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 818.675890] env[75863]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 818.676056] env[75863]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75863) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 820.683742] env[75863]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Session token - acquiring [ 820.684061] env[75863]: DEBUG networking_nsxv3.common.locking [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75863) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 820.684258] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 820.684386] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 820.684747] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 820.694011] env[75863]: DEBUG oslo_concurrency.lockutils [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75863) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 820.694243] env[75863]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 820.694385] env[75863]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75863) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 822.696892] env[75863]: CRITICAL neutron [None req-4766f39c-8b45-414b-8516-6170e43c04d2 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 822.696892] env[75863]: ERROR neutron Traceback (most recent call last): [ 822.696892] env[75863]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 822.696892] env[75863]: ERROR neutron sys.exit(main()) [ 822.696892] env[75863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 822.696892] env[75863]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 822.696892] env[75863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 822.696892] env[75863]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 822.696892] env[75863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 822.696892] env[75863]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 822.696892] env[75863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 822.696892] env[75863]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 822.696892] env[75863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 822.696892] env[75863]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 822.696892] env[75863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 822.696892] env[75863]: ERROR neutron response = self.get(path=path, params=params) [ 822.697587] env[75863]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 822.697587] env[75863]: ERROR neutron raise RuntimeError(msg, last_err) [ 822.697587] env[75863]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 822.697587] env[75863]: ERROR neutron [ 825.189573] env[75869]: INFO neutron.common.config [-] Logging enabled! [ 825.190085] env[75869]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 825.190085] env[75869]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75869) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 825.190286] env[75869]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 825.190380] env[75869]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 825.191225] env[75869]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 825.205002] env[75869]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Session token - acquiring [ 825.205763] env[75869]: DEBUG networking_nsxv3.common.locking [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75869) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 825.205966] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 825.206095] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 825.206448] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 825.216056] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 825.216344] env[75869]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 825.216522] env[75869]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75869) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 827.224452] env[75869]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Session token - acquiring [ 827.224814] env[75869]: DEBUG networking_nsxv3.common.locking [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75869) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 827.224886] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 827.225370] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 827.225370] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 827.235002] env[75869]: DEBUG oslo_concurrency.lockutils [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75869) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 827.235275] env[75869]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 827.235428] env[75869]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75869) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 829.238427] env[75869]: CRITICAL neutron [None req-ba1b48ea-e51c-45a8-88d9-5232708acfa3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 829.238427] env[75869]: ERROR neutron Traceback (most recent call last): [ 829.238427] env[75869]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 829.238427] env[75869]: ERROR neutron sys.exit(main()) [ 829.238427] env[75869]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 829.238427] env[75869]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 829.238427] env[75869]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 829.238427] env[75869]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 829.238427] env[75869]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 829.238427] env[75869]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 829.238427] env[75869]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 829.238427] env[75869]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 829.238427] env[75869]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 829.238427] env[75869]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 829.238427] env[75869]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 829.238427] env[75869]: ERROR neutron response = self.get(path=path, params=params) [ 829.239212] env[75869]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 829.239212] env[75869]: ERROR neutron raise RuntimeError(msg, last_err) [ 829.239212] env[75869]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 829.239212] env[75869]: ERROR neutron [ 831.672063] env[75873]: INFO neutron.common.config [-] Logging enabled! [ 831.672492] env[75873]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 831.672536] env[75873]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75873) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 831.672774] env[75873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 831.672870] env[75873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 831.673769] env[75873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 831.687208] env[75873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Session token - acquiring [ 831.687968] env[75873]: DEBUG networking_nsxv3.common.locking [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75873) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 831.688187] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 831.688318] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 831.688744] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 831.697481] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 831.697763] env[75873]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 831.697923] env[75873]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75873) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 833.705773] env[75873]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Session token - acquiring [ 833.706242] env[75873]: DEBUG networking_nsxv3.common.locking [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75873) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 833.706242] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 833.706411] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 833.706637] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 833.715943] env[75873]: DEBUG oslo_concurrency.lockutils [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75873) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 833.716228] env[75873]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 833.716385] env[75873]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75873) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 835.719045] env[75873]: CRITICAL neutron [None req-eddbd630-d52a-4386-8cdb-11ed0e60a6de None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 835.719045] env[75873]: ERROR neutron Traceback (most recent call last): [ 835.719045] env[75873]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 835.719045] env[75873]: ERROR neutron sys.exit(main()) [ 835.719045] env[75873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 835.719045] env[75873]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 835.719045] env[75873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 835.719045] env[75873]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 835.719045] env[75873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 835.719045] env[75873]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 835.719045] env[75873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 835.719045] env[75873]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 835.719045] env[75873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 835.719045] env[75873]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 835.719045] env[75873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 835.719045] env[75873]: ERROR neutron response = self.get(path=path, params=params) [ 835.720187] env[75873]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 835.720187] env[75873]: ERROR neutron raise RuntimeError(msg, last_err) [ 835.720187] env[75873]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 835.720187] env[75873]: ERROR neutron [ 838.161957] env[75879]: INFO neutron.common.config [-] Logging enabled! [ 838.162405] env[75879]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 838.162472] env[75879]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75879) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 838.162680] env[75879]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 838.162792] env[75879]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 838.163670] env[75879]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 838.177112] env[75879]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Session token - acquiring [ 838.177877] env[75879]: DEBUG networking_nsxv3.common.locking [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75879) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 838.178091] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 838.178189] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 838.178540] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 838.187408] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 838.187698] env[75879]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 838.187839] env[75879]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75879) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 840.195541] env[75879]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Session token - acquiring [ 840.195905] env[75879]: DEBUG networking_nsxv3.common.locking [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75879) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 840.196092] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 840.196166] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 840.196434] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 840.205862] env[75879]: DEBUG oslo_concurrency.lockutils [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75879) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 840.206102] env[75879]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 840.206245] env[75879]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75879) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 842.207462] env[75879]: CRITICAL neutron [None req-ebe4936c-366d-4520-941f-845dd51749f4 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 842.207462] env[75879]: ERROR neutron Traceback (most recent call last): [ 842.207462] env[75879]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 842.207462] env[75879]: ERROR neutron sys.exit(main()) [ 842.207462] env[75879]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 842.207462] env[75879]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 842.207462] env[75879]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 842.207462] env[75879]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 842.207462] env[75879]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 842.207462] env[75879]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 842.207462] env[75879]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 842.207462] env[75879]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 842.207462] env[75879]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 842.207462] env[75879]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 842.207462] env[75879]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 842.207462] env[75879]: ERROR neutron response = self.get(path=path, params=params) [ 842.208274] env[75879]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 842.208274] env[75879]: ERROR neutron raise RuntimeError(msg, last_err) [ 842.208274] env[75879]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 842.208274] env[75879]: ERROR neutron [ 844.696094] env[75885]: INFO neutron.common.config [-] Logging enabled! [ 844.696556] env[75885]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 844.696556] env[75885]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75885) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 844.696779] env[75885]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 844.696873] env[75885]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 844.697738] env[75885]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 844.710558] env[75885]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Session token - acquiring [ 844.711295] env[75885]: DEBUG networking_nsxv3.common.locking [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75885) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 844.711492] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 844.711636] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 844.712010] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 844.721869] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 844.722283] env[75885]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 844.722480] env[75885]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75885) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 846.730267] env[75885]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Session token - acquiring [ 846.730680] env[75885]: DEBUG networking_nsxv3.common.locking [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75885) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 846.730750] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 846.730885] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 846.731159] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 846.740367] env[75885]: DEBUG oslo_concurrency.lockutils [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75885) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 846.740627] env[75885]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 846.740809] env[75885]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75885) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 848.743760] env[75885]: CRITICAL neutron [None req-0b50b436-a9c4-48fd-bb91-bf881932932a None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 848.743760] env[75885]: ERROR neutron Traceback (most recent call last): [ 848.743760] env[75885]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 848.743760] env[75885]: ERROR neutron sys.exit(main()) [ 848.743760] env[75885]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 848.743760] env[75885]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 848.743760] env[75885]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 848.743760] env[75885]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 848.743760] env[75885]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 848.743760] env[75885]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 848.743760] env[75885]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 848.743760] env[75885]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 848.743760] env[75885]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 848.743760] env[75885]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 848.743760] env[75885]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 848.743760] env[75885]: ERROR neutron response = self.get(path=path, params=params) [ 848.744619] env[75885]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 848.744619] env[75885]: ERROR neutron raise RuntimeError(msg, last_err) [ 848.744619] env[75885]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 848.744619] env[75885]: ERROR neutron [ 851.170991] env[75889]: INFO neutron.common.config [-] Logging enabled! [ 851.171496] env[75889]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 851.171496] env[75889]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75889) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 851.171693] env[75889]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 851.171790] env[75889]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 851.172667] env[75889]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 851.185781] env[75889]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Session token - acquiring [ 851.186510] env[75889]: DEBUG networking_nsxv3.common.locking [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75889) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 851.186712] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 851.186835] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 851.187175] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 851.196284] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 851.196550] env[75889]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 851.196713] env[75889]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75889) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 853.205333] env[75889]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Session token - acquiring [ 853.205684] env[75889]: DEBUG networking_nsxv3.common.locking [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75889) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 853.205853] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 853.205900] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 853.206173] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 853.215944] env[75889]: DEBUG oslo_concurrency.lockutils [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75889) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 853.216273] env[75889]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 853.216452] env[75889]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75889) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 855.219404] env[75889]: CRITICAL neutron [None req-aa12a41d-b8fa-4f11-ba8a-542a2f2d6ddf None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 855.219404] env[75889]: ERROR neutron Traceback (most recent call last): [ 855.219404] env[75889]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 855.219404] env[75889]: ERROR neutron sys.exit(main()) [ 855.219404] env[75889]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 855.219404] env[75889]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 855.219404] env[75889]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 855.219404] env[75889]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 855.219404] env[75889]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 855.219404] env[75889]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 855.219404] env[75889]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 855.219404] env[75889]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 855.219404] env[75889]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 855.219404] env[75889]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 855.219404] env[75889]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 855.219404] env[75889]: ERROR neutron response = self.get(path=path, params=params) [ 855.220260] env[75889]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 855.220260] env[75889]: ERROR neutron raise RuntimeError(msg, last_err) [ 855.220260] env[75889]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 855.220260] env[75889]: ERROR neutron [ 857.667719] env[75937]: INFO neutron.common.config [-] Logging enabled! [ 857.668162] env[75937]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 857.668289] env[75937]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75937) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 857.668555] env[75937]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 857.668655] env[75937]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 857.669483] env[75937]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 857.682819] env[75937]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Session token - acquiring [ 857.683533] env[75937]: DEBUG networking_nsxv3.common.locking [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75937) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 857.683733] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 857.683860] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 857.684293] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 857.693395] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 857.693629] env[75937]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 857.693775] env[75937]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75937) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 859.701719] env[75937]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Session token - acquiring [ 859.702108] env[75937]: DEBUG networking_nsxv3.common.locking [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75937) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 859.702243] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 859.702323] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 859.702608] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 859.712115] env[75937]: DEBUG oslo_concurrency.lockutils [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75937) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 859.712465] env[75937]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 859.712635] env[75937]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75937) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 861.715582] env[75937]: CRITICAL neutron [None req-3c26bc65-d49a-4702-8b5a-71bfafa35ad3 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 861.715582] env[75937]: ERROR neutron Traceback (most recent call last): [ 861.715582] env[75937]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 861.715582] env[75937]: ERROR neutron sys.exit(main()) [ 861.715582] env[75937]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 861.715582] env[75937]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 861.715582] env[75937]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 861.715582] env[75937]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 861.715582] env[75937]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 861.715582] env[75937]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 861.715582] env[75937]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 861.715582] env[75937]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 861.715582] env[75937]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 861.715582] env[75937]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 861.715582] env[75937]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 861.715582] env[75937]: ERROR neutron response = self.get(path=path, params=params) [ 861.716455] env[75937]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 861.716455] env[75937]: ERROR neutron raise RuntimeError(msg, last_err) [ 861.716455] env[75937]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 861.716455] env[75937]: ERROR neutron [ 864.171581] env[75941]: INFO neutron.common.config [-] Logging enabled! [ 864.171923] env[75941]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 864.172087] env[75941]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75941) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 864.172316] env[75941]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 864.172413] env[75941]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 864.173254] env[75941]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 864.186749] env[75941]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Session token - acquiring [ 864.187506] env[75941]: DEBUG networking_nsxv3.common.locking [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75941) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 864.187702] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 864.187829] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 864.188193] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 864.197282] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 864.197538] env[75941]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 864.197688] env[75941]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75941) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 866.204309] env[75941]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Session token - acquiring [ 866.204659] env[75941]: DEBUG networking_nsxv3.common.locking [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75941) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 866.204829] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 866.204929] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 866.205208] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 866.215087] env[75941]: DEBUG oslo_concurrency.lockutils [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75941) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 866.215370] env[75941]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 866.215534] env[75941]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75941) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 868.218486] env[75941]: CRITICAL neutron [None req-f98e4e12-c13c-41d1-8946-1237069322ee None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 868.218486] env[75941]: ERROR neutron Traceback (most recent call last): [ 868.218486] env[75941]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 868.218486] env[75941]: ERROR neutron sys.exit(main()) [ 868.218486] env[75941]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 868.218486] env[75941]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 868.218486] env[75941]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 868.218486] env[75941]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 868.218486] env[75941]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 868.218486] env[75941]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 868.218486] env[75941]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 868.218486] env[75941]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 868.218486] env[75941]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 868.218486] env[75941]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 868.218486] env[75941]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 868.218486] env[75941]: ERROR neutron response = self.get(path=path, params=params) [ 868.219275] env[75941]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 868.219275] env[75941]: ERROR neutron raise RuntimeError(msg, last_err) [ 868.219275] env[75941]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 868.219275] env[75941]: ERROR neutron [ 870.718553] env[75947]: INFO neutron.common.config [-] Logging enabled! [ 870.718848] env[75947]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 870.718966] env[75947]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75947) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 870.719163] env[75947]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 870.719268] env[75947]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 870.720193] env[75947]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 870.740201] env[75947]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Session token - acquiring [ 870.740996] env[75947]: DEBUG networking_nsxv3.common.locking [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75947) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 870.741210] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 870.741338] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 870.741704] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 870.751094] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 870.751334] env[75947]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 870.751475] env[75947]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75947) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 872.760053] env[75947]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Session token - acquiring [ 872.760053] env[75947]: DEBUG networking_nsxv3.common.locking [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75947) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 872.760517] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 872.760517] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 872.760517] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 872.769812] env[75947]: DEBUG oslo_concurrency.lockutils [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75947) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 872.770062] env[75947]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 872.770206] env[75947]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75947) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 874.773165] env[75947]: CRITICAL neutron [None req-e17c97de-a3c0-4119-9e11-293795ac4987 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 874.773165] env[75947]: ERROR neutron Traceback (most recent call last): [ 874.773165] env[75947]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 874.773165] env[75947]: ERROR neutron sys.exit(main()) [ 874.773165] env[75947]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 874.773165] env[75947]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 874.773165] env[75947]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 874.773165] env[75947]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 874.773165] env[75947]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 874.773165] env[75947]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 874.773165] env[75947]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 874.773165] env[75947]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 874.773165] env[75947]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 874.773165] env[75947]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 874.773165] env[75947]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 874.773165] env[75947]: ERROR neutron response = self.get(path=path, params=params) [ 874.774035] env[75947]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 874.774035] env[75947]: ERROR neutron raise RuntimeError(msg, last_err) [ 874.774035] env[75947]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 874.774035] env[75947]: ERROR neutron [ 877.150662] env[75953]: INFO neutron.common.config [-] Logging enabled! [ 877.151085] env[75953]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 877.151204] env[75953]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75953) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 877.151369] env[75953]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 877.151465] env[75953]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 877.152330] env[75953]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 877.165804] env[75953]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Session token - acquiring [ 877.166722] env[75953]: DEBUG networking_nsxv3.common.locking [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75953) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 877.167162] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 877.167472] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 877.167988] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 877.176844] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 877.177263] env[75953]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 877.177602] env[75953]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75953) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 879.185539] env[75953]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Session token - acquiring [ 879.186400] env[75953]: DEBUG networking_nsxv3.common.locking [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75953) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 879.186824] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 879.187111] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 879.187541] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 879.196709] env[75953]: DEBUG oslo_concurrency.lockutils [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75953) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 879.197096] env[75953]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 879.197425] env[75953]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75953) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 881.200500] env[75953]: CRITICAL neutron [None req-13426e1d-2e4b-43df-9a64-0a71f53b6283 None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 881.200500] env[75953]: ERROR neutron Traceback (most recent call last): [ 881.200500] env[75953]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 881.200500] env[75953]: ERROR neutron sys.exit(main()) [ 881.200500] env[75953]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 881.200500] env[75953]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 881.200500] env[75953]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 881.200500] env[75953]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 881.200500] env[75953]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 881.200500] env[75953]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 881.200500] env[75953]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 881.200500] env[75953]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 881.200500] env[75953]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 881.200500] env[75953]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 881.200500] env[75953]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 881.200500] env[75953]: ERROR neutron response = self.get(path=path, params=params) [ 881.201282] env[75953]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 881.201282] env[75953]: ERROR neutron raise RuntimeError(msg, last_err) [ 881.201282] env[75953]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 881.201282] env[75953]: ERROR neutron [ 883.660974] env[75957]: INFO neutron.common.config [-] Logging enabled! [ 883.661423] env[75957]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 883.661467] env[75957]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=75957) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 883.661660] env[75957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 883.661750] env[75957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 883.662586] env[75957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 883.676487] env[75957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Session token - acquiring [ 883.677198] env[75957]: DEBUG networking_nsxv3.common.locking [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75957) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 883.677394] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 883.677549] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 883.677901] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 883.686776] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 883.687021] env[75957]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 883.687165] env[75957]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=75957) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 885.693733] env[75957]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Session token - acquiring [ 885.694051] env[75957]: DEBUG networking_nsxv3.common.locking [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=75957) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 885.694259] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 885.694383] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 885.694651] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 885.704220] env[75957]: DEBUG oslo_concurrency.lockutils [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=75957) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 885.704495] env[75957]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 885.704650] env[75957]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Retrying connection (2/2) with timeout 2s for Client.get {{(pid=75957) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}} [ 887.707612] env[75957]: CRITICAL neutron [None req-8b29475e-772b-4ba4-a8b8-accbb61b198e None None] Unhandled error: RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 887.707612] env[75957]: ERROR neutron Traceback (most recent call last): [ 887.707612] env[75957]: ERROR neutron File "/usr/local/bin/neutron-nsxv3-agent", line 10, in [ 887.707612] env[75957]: ERROR neutron sys.exit(main()) [ 887.707612] env[75957]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 331, in main [ 887.707612] env[75957]: ERROR neutron NSXv3Manager(rpc=nsxv3_rpc.NSXv3ServerRpcApi()), [ 887.707612] env[75957]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 145, in __init__ [ 887.707612] env[75957]: ERROR neutron self.mngr_provider = provider_nsx_mgmt.Provider() [ 887.707612] env[75957]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 443, in __init__ [ 887.707612] env[75957]: ERROR neutron super(Provider, self).__init__(client=Client(), zone_id=None) [ 887.707612] env[75957]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py", line 207, in __init__ [ 887.707612] env[75957]: ERROR neutron self.zone_id, self.zone_tags = self._load_zones() [ 887.707612] env[75957]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py", line 454, in _load_zones [ 887.707612] env[75957]: ERROR neutron for zone in self.client.get_all(path="/api/v1/transport-zones"): [ 887.707612] env[75957]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 276, in get_all [ 887.707612] env[75957]: ERROR neutron response = self.get(path=path, params=params) [ 887.708832] env[75957]: ERROR neutron File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py", line 129, in decorator [ 887.708832] env[75957]: ERROR neutron raise RuntimeError(msg, last_err) [ 887.708832] env[75957]: ERROR neutron RuntimeError: ('Retrying connection (2/2) with timeout 2s for Client.get', HTTPError('403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create')) [ 887.708832] env[75957]: ERROR neutron [ 890.252052] env[76067]: INFO neutron.common.config [-] Logging enabled! [ 890.252454] env[76067]: INFO neutron.common.config [-] /usr/local/bin/neutron-nsxv3-agent version 0.0.1.dev46 [ 890.252565] env[76067]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=76067) setup_logging /opt/stack/neutron/neutron/common/config.py:123}} [ 890.252802] env[76067]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ... [ 890.252898] env[76067]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed. [ 890.253768] env[76067]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Looking for TransportZone with name nsx-vlan-transportzone. [ 890.267801] env[76067]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Session token - acquiring [ 890.268600] env[76067]: DEBUG networking_nsxv3.common.locking [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [>, , , , , , , , , ] {{(pid=76067) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}} [ 890.268868] env[76067]: DEBUG oslo_concurrency.lockutils [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76067) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} [ 890.269011] env[76067]: DEBUG oslo_concurrency.lockutils [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76067) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} [ 890.269368] env[76067]: DEBUG oslo_concurrency.lockutils [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76067) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:323}} [ 890.278578] env[76067]: DEBUG oslo_concurrency.lockutils [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=76067) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} [ 890.278816] env[76067]: ERROR networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Request=Function GET Argumetns {'path': '/api/v1/transport-zones', 'params': {'page_size': 200, 'cursor': ''}} Response=403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create: requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nsxt.osci.c.eu-de-1.cloud.sap:443/api/session/create [ 890.279043] env[76067]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-dbf05785-1d31-4b0d-806f-9a96944b0053 None None] Retrying connection (1/2) with timeout 2s for Client.get {{(pid=76067) decorator /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py:126}}